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Method and (System for Rating Users, User^Pdntributed Items, 
Groupings of Related User-Contributed Items and Other 

Items 

Related Applications g 

5 This application is related to and claims priority under 35 U.S.C. § 119(e) 

from U.S. Provisional Application No. 60/167,594 filed November 26, 1999, which is 
herein incorporated by reference. 

A. Technical Field 

This application relates to networks, such as computer networks and more 
10 specifically to a method and system for rating users, user-contributed items, groupings 
of related user-contributed items and other items on a network. 

B. Background 

^ A significant and distinctive feature of wide-area networks, including global 
wide area networks such as the Internet, is sometimes considered to be the ability of 

15 users anywhere on the network to access a centralized source for content in a 
particular category. The provision of information from centralized sources has a long 
tradition in print and electronic media. However, in many respects, despite the wider 
possibilities for dissemination, this is an "old media" concept, based on the economics 
of content production and information distribution before the advent of global wide 

20 area networks. Remarkably, this model still holds considerable sway, despite 
overwhelming changes in the technology and economics of information. More than 
ever before, any user anywhere has the ability not only to access centrally produced 
content, but also to interact with other users anywhere — at almost zero marginal cost. 

Nevertheless, several considerations have reduced the extent and growth of 
25 decentralized interactions over wide-area networks. -If is difficult for a user to collect 
accurate information and form a meaningful opinion about the competence and 
credibility of another party without extensive contact — costly in time and effort. 
When interactions do occur, they are largely one-time or limited in number, with few 
expectations of repeat dealings. There is little incentive for users to share their 
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assessments of^Kr users and such assessments cannoWKcessarily be taken at face 
value. 

Accessing raw, decentralized content that is time sensitive, without the ability 
to identify which users regularly produce items of higher Quality, means wading 
5 through an avalanche of biased, narrow or uninformed material obscuring valuable 
contributions. Even with the passage of time, after other users have reviewed the 
content and reacted to it, there may be more information to assess rather than less. 
User comments are rarely communicated in a form useable by others to filter content 
for Quality. By sheer volume, commentary is typically focused on controversial or 
10 negative content that has (often intentionally) prompted a strong reaction. 

Any system that rates Quality or content must achieve a critical mass of input 
in order to produce useful output. However, in the case of the Internet and other 
wide-area networks, ratings are systemically underprovided. The problem is 
particularly acute for content that is produced decentrally in a networked environment. 
15 When rating schemes seek input widely and make results freely available on public 
networks, users do not have appropriate incentives to contribute. 

Although users may engage in some rating, the possibility to "free ride" on the 
contributions of others discourages involvement. Users may respond to financial 
incentives to provide ratings, but it is difficult to monitor the Quality of these 
20 contributions. Also, if such incentives are financed through user fees restricting 
access, fewer users benefit from the ratings. 

Much of the prior art in the field of ratings has to do with predicting what is 
the best item of content to deliver next to a particular user — often for strictly 
commercial purposes — based on the user's revealed preferences and current position 
25 on the network, so-called "collaborative filtering." The mechanism of this approach is 
generally to associate each user with other users of like mind who have been given 
appropriate incentives to provide ratings, or who have otherwise provided ratings 
incidentally or voluntarily. 
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Such a raithod does little, however, to assist rRrs exploring new subject 
matter. The emphasis is not absolute item Quality, but the relative likes and dislikes 
of different people and the objective is to provide a user who has revealed certain 
preferences with more of the same. At best, such a method determines how the user, 
5 with the user's current breadth of knowledge and experience, might rate items. It does 
little to identify experts acknowledged by other people who have themselves 
demonstrated talent. 

The inadequacies of conventional rating systems have exacerbated the 
centralized nature of content production and filtering on the Internet. Groups of 
10 online "experts," such as the staff or affiliates of Internet portals or community web 
sites with tight, centralized editorial control, play an excessive role. As a result, fewer 
and less meaningful interactions occur that take advantage of the rich, decentralized 
resources of wide-area networks. 

Summary of Preferred Embodiments 

15 The described embodiments of the present invention offer alternative 

structures for decentralized interactions among users on wide-area networks and a 
method of constructing, applying and distributing ratings of users, user-contributed 
items, groupings of related user-contributed items and other items. 

In certain of the described embodiments, the higher the Quality of the items 
20 contributed by or associated with a particular user, the more weight assigned to ratings 
supplied by such user. At the same time, a calculation using weighted ratings 
determines item Quality. In effect, the weights assigned to ratings affect Quality and 
Quality affects the weights assigned to ratings, simultaneously. The preferred 
embodiments include a solution to the "circular" character of this approach. 

25 The described embodiments include a series of mathematical methods 

embedded in network processes. These network processes provide for a series of 
interactions between user nodes, mediated by other network elements that create a 
structured environment. Among other things, user interactions can include 
participation in a web site's online discussion group or chat facilities, an e-mail based 
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mailing list, unidnWKbnal, bidirectional or widely broadcSirdigital video or audio 
communication, a distributed communication facility such as Usenet newsgroups or 
IRC chat, or an online auction or other facility through which users provide, or agree 
to provide, goods and services in commerce, in each case enhanced with features of 
5 the described embodiments. 

Among such enhancements are facilities that provide participants with an 
opportunity to rate each instance of one user interacting with other users, referred to 
herein as an "item." For example, a chat session would generally be considered as a 
separate item for each user who participated, for example by injecting at least one 
10 statement into the session during a particular hour. As another example, particular 
charities might be considered to be items. Users can post their ratings of the charities, 
and the charities are assigned a Quality value as described below. 

In certain preferred embodiments, ratings of items contributed by or associated 
with a particular user are used to construct measurements of the user's competence 
1 5 and credibility as a participant in a networked environment, called herein "Expertise" 
and "Regard." In general, the term "Expertise" is used in connection with the 
"Expert" methods described herein and the term "Regard" is used in connection with 
the "High Regard" methods described herein. 

Either Expertise or Regard can be used as a factor in determining the relative 
20 weight assigned to each user's evaluations of items contributed by or associated with 
other users, producing a measure of the relevance, accuracy and importance of a 
particular item, referred to herein as item "Quality." For example, the Expertise or 
Regard of users rating a particular discussion group posting might be used to weight 
their ratings in a calculation of the posting' s Quality. 

25 Either Expertise or Regard may also be used to predict the likely relevance, 

accuracy and importance of a user's contributed items (both historical and future), for 
example, when a particular item is too new to have itself received ratings or when 
ratings of a particular item are sparse. 
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relevant to an evaluation of the user for other purposes, including purposes not 
associated with interactions across wide-area networks. 

Expertise, Regard or Quality, or related measures, may be bundled with 
5 relevant content and transmitted to nodes on a network, offering multiple additional 
opportunities to enhance user interactions, in some embodiments. One or more of 
these measures may also be used to filter, highlight, sort or otherwise evaluate items 
or to limit a user's interactions to items and other users who meet minimum standards. 



10 interject negative, disruptive or misleading items into the system through 
incompetence or ill will, can be identified, downgraded or even removed from view. 
Equally, items by users who regularly provide items of high Quality can be given 
immediate prominence, even before receiving a single user rating. 

High ratings can raise the profile of any item, irrespective of the Expertise or 
1 5 Regard of the user who contributed it. Also, an item that misses the mark and receive 
poor ratings can be identified, downgraded or removed from view, even if the user 
who contributed it has high Expertise or Regard. 

Arguably, these methods benefit not only users in general, but also the 
contributors, who would want more visibility for their best items. 

20 The described embodiments of applicants' invention also include methods 

enabling users to "Vouch" for items contributed by another user by associating their 
own Expertise or Regard with such items. In this manner, a well Regarded user (also 
called a highly Regarded user) can quickly raise the prominence of a valuable item 
contributed by an unknown or otherwise poorly Regarded user. Similarly, users are 

25 enabled to "Discredit" items contributed by other users by asserting their own 
Expertise or Regard in opposition to such items. In these cases, the Expertise or 
Regard not only of the contributor, but also of the user who Vouched for or 
Discredited such items can be enhanced or diminished depending on other user's 
ratings of such items. 



Items contributed by users who regularly contribute items of poor Quality or 
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Lgs current and relevant, there are alsWPTethods to remove ratings 




from the calculation of Expertise or Regard over time, based on the age of items, the 
Expertise or Regard of the user and other factors. 

To facilitate the accumulation of a critical mass of data, certain embodiments 
5 will construct an environment in which users provide ratings as a natural part of 
navigating through items. For example, if the user views an item for a period of time 
suggesting the user has given it attention and consideration, the user may be 
encouraged or required either to provide a rating or to exit the system, or the portion 
of the system the user is currently interacting with. Also, for example: the act of 

10 moving from one item to another at a particular moment, when certain visual cues 
(among a rotating series of alternatives) appear on the user interface, may indicate that 
the user has selected a particular rating. The intention is to create an efficient manner 
for users to provide ratings, to limit the requirement to provide ratings to situations in 
which the user has actually reviewed the item and to generate a large body of useful 

15 rating data. By providing an efficient user interface and striking an appropriate 
balance, the breadth, Quality and utility of systematic ratings- more than compensates 
users for the additional effort required to supply ratings. 

Advantages of the described embodiments will be set forth in part in the 
description which follows and in part will be obvious from the description, or may be 
20 learned by practice of the described embodiments. The objects and advantages of the 
described embodiments will be realized and attained by means of the elements and 
combinations particularly pointed out in the appended claims and equivalents. 



Fig. 2(b) describes alternative rating schemes that can be used in accordance 
with the present invention. 



Brief Description of the Drawings 



Fig. 1 shows a network, various users of the network and a network server. 



25 



Fig. 2(a) shows an example of a user interface allowing users to view and rate 



items. 
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Fig. 2(cj^Bws a web page incorporating a ratin^Kheme in accordance with 
the present invention. 



Fig. 3 shows a simple example in a system having only four users 
demonstrating that ratings supplied by users are not necessarily valued equally. 

5 Fig. 4(a) and 4(b) show a simple example demonstrating that a user's Regard 

is affected by how other users rate the items that the user contributes and by the 
Regard of those users, and that one user's Regard can affect the Regard of other users 
and the Quality of items contributed by other users. 

Figs. 5(a) and 5(b) show another simple example demonstrating that a user's 
10 Regard is affected by how other users rate the items that the user contributes and by 
the Regard of those users, and that a user's Regard can affect the Regard of other 
users and the Quality of items contributed by other users. 

Figs. 6(a) and 6(b) show a simple example demonstrating that a user's 
Expertise is affected by how others rate the items that the user contributes and that a 
15 user's Expertise can affect the Quality of items contributed by other users. 

Fig. 7(a) is an overview of data flows in an example High Regard ratings 
system, showing the circular nature of Regard affecting the Regard of other users and 
the Quality of items contributed by other users. 

Fig. 7(b) is a more detailed overview of data flows in an example High Regard 
20 ratings system, showing the circular nature of Regard affecting the Regard of other 
users and the Quality of items contributed by other users, emphasizing procedures for 
the flow, storage and interpretation of data. . 

Figs. 7(c)-7(e) are flow charts showing the inputs and outputs to "basic" 
determinations of item Quality, user Regard and user Expertise. 

25 Fig. 8 is a plot of a Regard Inertial Model. 

Fig. 9(a) is a plot of a Quality Inertia Model. 

Fig. 9(b) is a plot of a Segmented Decay Model. 
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Fig. 10 sh& 




example of a small Regard data sel 




Fig. 1 1 shows an example of a data structure used to store and retrieve data 
required to perform calculations of Regard and Quality in a preferred embodiment of 
the present invention. 



including a rating server. 

Fig. 12(b) is a block diagram of another example forum server application 
including a rating server, where the rating information is sent directly to a user's web 
page. 



diagram of Fig. 12(b). 

Fig. 13 is a block diagram of another example forum server application 
communicating with a separate communication forum. 

Fig. 14 is a block diagram of another example forum server application 
1 5 communicating with a separate communication forum and an ad server. 

Fig. 15(a) is a flow diagram showing communication between elements of Fig. 
14 during forum/thread index intercommunication. 

Fig. 15(b) is a flow diagram showing communication between elements of Fig. 
14 during article view intercommunication. 

20 Fig. 15(c) is a flow diagram showing communication between elements of Fig. 

14 during post/reply intercommunication. 

Figs. 16(a)- 16(h) are flow charts showing methods used during the 
intercommunication processes of Figs. 15(a)- 15(c). 



5 



Fig. 12(a) is a block diagram of a first example forum server application 



10 



Fig. 12(c) is an example of a web page generated in accordance with the block 



Fig. 17 is a block diagram of an example e-commerce server. 



25 



Fig. 1 8 is a block diagram of an example auction server and a rating server. 
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Fig. 19 block diagram of an example s^^R* for an individual and 
commercial rating service. 



Figs. 20(a) and 20(b) are, respectively, a graph and a table that aid in 
explaining the concepts of "Vouching" and "Discrediting." 

5 Detailed Description of Embodiments 

1 . Overview 

Fig. 1 shows a network 100, such as the Internet, an intranet, a wide-area 
network (WAN), a wireless or telephonic network, or any other appropriate network. 
Network 100 can also be a combination of various types of networks or of various 

10 networks and sub-networks. Users communicate via the network 100 by sending 
information by way of methods and protocols appropriate to the network 100. 
Although it will be understood that many users can access network 100 
simultaneously, three users 110, 120 and 130 are shown. A user can be a human 
being or another entity, such as a computer program capable of accessing network 

15 100. Fig. 1 also shows a rating server 140 as discussed below. 

In the network of Fig. 1 , users contribute items, such as, for example, e-mail 
messages or discussion group postings. (As discussed below in a separate section, the 
term "items" encompass a wide variety of things.) The users view each other's items 
and some or all of the users rate the items contributed by other users. For example, in 

20 Fig. 1, user 110 reads other items and rates them, but does not contribute items 
himself. In Fig. 1, user 120 reads others' items but does not rate them. User 120 
does, however, contribute items. In Fig. 1, user 130 reads others' items and rates 
them. User 130 also contributes items. The three users discussed herein are intended 
to show that various users interact with network 100 in different ways. 

25 In Fig. 1, user 110 receives items 112 contributed by other users, including the 

Quality of the items and the Regard of the users who contributed the items. It should 
be understood that, although Fig. 1 discusses a system using the Regard of users, Fig. 
1 could also apply to a system that uses the Expertise of users (as both terms are 
defined herein). Only Regard is shown in the example of Fig. 1 to aid in maintaining 

30 the simplicity of the example. User 110 contributes a rating 1 14 of one or more of the 
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reWives items 122 contributed by other uscts, 



items. User 120 reWTves items 122 contributed by other uscts, including the Quality 
of the items and the Regard of the users who contributed the items. User 120 
contributes one or more items 126 of his own. User 130 receives items 132 
contributed by other users, including the Quality of the items and a Regard of the 
5 users who contributed the items. User 130 contributes a rating 134 of one of more of 
the items. User 130 contributes one of more items 136. 

It will be understood that Fig. 1 is provided by way or example and not 
limitation. The invention can be implemented in a network as shown, or in other 
environments, such as, for example, a database in which users enter, read and access 

10 items in the database. The invention could also be implemented in, for example, an e- 
mail system operating in a networked environment. It will be understood that, in the 
described embodiments, the functionality described herein is preferably performed by 
a data processing system or systems performing instructions stored on a medium or 
memory accessible by the data processing system(s). The invention is not limited to 

1 5 the architecture, programming models, protocols or procedures shown and described 
herein. 

Fig. 2(a) shows an example of a user interface allowing users to view and rate 
items. Although not shown, users can also contribute their own items. Fig. 2(a) 
shows a web page that displays messages in an online discussion forum. The text of a 
20 current message is displayed in an area 202. As the user views a current message, he 
can rate the message using rating area 204 (or a similar rating scheme, such as one of 
the alternative ratings schemes described below). 

The user interface of Fig. 2(a) uses alternating visual cues. Specifically, in the 
ratings scheme shown in the Figure, five diamond boxes 204 are displayed. One of 

25 these diamonds is highlighted per second, with the highlights preferably moving 
sequentially from left to right and wrapping from diamond #5 back to diamond #1. 
The pattern repeats until a predefined user action is detected. A predefined user action 
can be, but is not limited to a keystroke, mouse movement or click, voice command 
and/or another appropriate command. When a predefined user action is detected, it 

30 selects a rating corresponding to a currently highlighted diamond in area 204. In 
certain embodiments, such a predefined user action also causes the user to proceed to 
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the next item, SBid, or elsewhere. For example, th^Kr clicks on a rating and 
automatically advances to a next item, thread, etc. 

The rating diamonds preferably correspond to varying ratings, from, for 
example, lowest to highest ratings. Thus, the user merely needs to wait until a rating 
5 that he agrees with is highlighted and perform one of the predefined user actions, such 
as slightly moving his mouse or touching any key on the keyboard. The user does not 
have to move a mouse controlled pointer to a particular location on the web page, or 
click or highlight specific objects or locations on the web page to rate messages, since 
the act of rating automatically causes the display to advance to the next message. 

10 In a preferred embodiment, the interface of Fig. 2(a), including diamonds 204, 

is implemented via an applet on the web page. The interface shown could be 
implemented using any appropriate method, such as via a browser plug-in, java code 
transmitted during a particular network connection and persisting for a limited period 
of time, or a standard html web page. 

15 Fig. 2(b) describes alternative rating schemes that can be used in accordance 

with the present invention. While Fig. 2(b) describes the "single click rating" scheme 
of Fig. 2(a), it also describes several other rating schemes. These include a "rate 
while you navigate" scheme and a "single keystroke/click rate/Navigate" scheme. In 
the rate while you navigate scheme, use of keyboard or mouse commands to navigate 

20 to other items while the visual cues are alternating is interpreted as selecting the rating 
represented by the currently highlighted visual cue ("rate while you navigate"). In the 
single keystroke/click rate/navigate scheme, the rating diamonds 242 are not 
automatically highlighted. Instead, the user clicks on an appropriate diamond or 
presses one of keys 1, 2, 3 of 4 on the keypad (or some other predetermined keys) to 

25 both select a rating for the current message and to go to the next article, next thread, 
out or elsewhere. 

Although not shown, other ratings schemes do not automatically advance the 
user to a next view, but require the user to perform the rating and navigation functions 
separately (even though one or the other can still be accomplished by alternating 
30 indicators as discussed above). For example, an alternative rating scheme places the 
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ie on the page. Still 



another scheme places the rating diamonds next to the messages. 

A message "thread" is a group of related messages having a linear order and a 
hierarchy of levels of indentation (representing interrelationship) so that a user can 
5 view the next or previous item in the message thread. It is important to note that, in 
the described embodiment, it is the messages, message threads, or other items that are 
being rated, not the authors/contributors of the items. Still another scheme allows the 
user to type a rating into a box on the web page or enter a rating in a drop-down menu 
or special window. Still another rating scheme allows the user to move a slider bar or 
10 similar non-discrete input device on a web page and translates the user's action into a 
numeric rating. Thus, the above user interfaces could also be implemented as non- 
discrete interfaces wherein user ratings fall along a spectrum (for example, between 0 
and 1) and are not limited to predetermined discrete values. 



1 5 the present invention. In this example, the ratings diamonds invite the user to rate the 
entire page, not just the contents of certain areas on the page. The rating diamonds 
can alternate as shown above or require the user to click on or enter an appropriate 
rating. 



20 transactions completed through a networked commerce facility, such as an online 
auction service. In a transaction-rating scheme, the item can be, for example, a 
completed transaction in which the user was the buyer or seller or another user 
monitoring the quality of the participation by the buyer or seller, or of the goods or 
services that are the subject of the transaction. Other types of items can also be rated 

25 in such a system. In such a rating scheme, Quality will attach to various transactions 
from a buyer's perspective and a seller's perspective. For example the buyer may rate 
the seller highly, but the seller may rate the buyer much lower. Thus, the same 
transaction receives very different ratings from buyer and seller, both of whom are 
"the author" of the transaction in some sense. Alternatively, the buyer behavior and 

30 the seller behavior can be considered two separate items that are rated separately. In 



Fig. 2(c) shows a web page incorporating a rating scheme in accordance with 



Still another rating scheme rates products and services offered on and 
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tay be able to provide a 



rating or only a limited number of users may provide ratings. 

Fig. 2(a) also shows a Regard diamond 206 (which represents the Regard of 
the author of the current item) and a Quality diamond 208 (which represents the 
5 Quality value assigned to the current item). In general, Regard can be used alone as 
an independent benchmark, as a predictor of the value of the user's historical and 
future contributions or, in a system-wide calculation, as a factor determining the 
relative weight assigned to the user's evaluations of other user's contributions. The 
Quality of an item is based on the ratings assigned to the item by other users and also 
10 based on the Regards of those other users. A discussion of the meaning of and 
preferred methods of obtaining Regard and Quality are discussed below in detail. It 
should be understood that an interface similar to that of Fig. 2(a) could also be used to 
display item Quality and user Expertise in a system that uses Expertise values rather 
than Regard values. 

15 Fig. 2(c) also shows a ranking diamond 244. This diamond represents the 

Quality of the web page. In this example, no web page author has been identified. 
This example demonstrates that Quality can be displayed for an item even though the 
item is not associated with a user. Alternately, both Regard (of the author) and 
Quality (of the page) could be displayed for the web page, in a manner similar to that 

20 shown in Fig. 2(a). Expertise could be used in place of Regard. 

Figs. 7(c)-7(e) are flow charts showing the inputs and outputs to "basic" 
determinations of item Quality, user Regard and user Expertise. In certain 
embodiments, each user is assigned a Regard value, in accordance with ratings that 
other users assign to items contributed by that user and in further accordance with the 

25 Regard values of the users who contribute the ratings. In certain other embodiments, 
each user is assigned an Expertise value, in accordance with ratings that other users 
assign to items contributed by that user. In certain preferred embodiments, a user is 
assigned a default Expertise or Quality value. Quality values are determined for each 
item contributed by a user, whether the systems implements Regard or Expertise. The 

30 Quality value of an item is determined in accordance with the ratings received for the 
item and the Regard (or Expertise) of the users who contributed the ratings. Details of 
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the determination* "basic" Quality, Regard and Expemse are discussed below in 
Section 5 et seq. Variations on the basic determinations are also discussed below in 
Section 5 et seq. 

Figs. 3-6 show a simple example in a system having only four users 
5 demonstrating that ratings supplied by users are not necessarily valued equally in a 
system using Regard. In the following examples, Figs. 3-5 show a system 
implementing Regard values. Fig. 6 shows a system implementing Expertise values. 
To enhance the clarity of the examples in Figs. 3-6, it is assumed that the system has 
only four users and that only one item is contributed and rated at a time. It is 
10 contemplated that most systems employing an embodiment of the present invention 
will have many more users and that multiple item contributions and multiple ratings 
by the users of other user's items will overlap in time. 

In Fig. 3, User 4 contributes Item A, which is viewed and rated by Users 1, 2 
and 3. In the example, User 1 is well Regarded (having, e.g., a Regard of 0.9). User 2 
15 is poorly Regarded, having a Regard of 0.1. User 3 is also poorly Regarded, having a 
Regard of 0.1. (Each User's Regard is based at least in part on his previous 
contributions and the ratings of other users for those contributions, as discussed below 
in detail). 

This example assumes that possible ratings fall between 0 and 1 inclusive. 

20 Other systems may, of course, use other ratings scales. In Fig. 3, well Regarded User 
1 "loves" Item A, giving it a rating of 1.0 (the highest rating possible). Poorly 
Regarded User 2 "hates" Item A, giving it a rating of 0.0 (the lowest rating possible). 
Poorly Regarded User 3 also "hates" Item A, giving it a rating of 0.0. Even though 
two users hate Item A, both of those users are poorly Regarded. Thus, Item A 

25 receives a Quality value that is closer to the rating by well Regarded User 1 than to 
the ratings received from User's 2 and 3. Thus, in a system using Regard, the ratings 
contributed by all users are not equally valued. Ratings from well Regarded users are 
valued more when determining the Quality of an item. It should be noted that the 
terms "love" and "hate" are presented here as an aid to understanding. The 

30 embodiment described here determines numeric ratings. For example, a user interface 
may allow a user to choose "love" or "hate" or some other word suggesting a reaction 
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somewhere in betwfen, but these non-numeric choices are^ventually translated into 
numeric values for the purpose of calculations in the preferred embodiments. 

Figs. 4(a) and 4(b) show a simple example in a system having only four users 
demonstrating that a user's Regard is affected by how others rate the items that the 
5 user contributes and by the Regard of those users and that a user's Regard can affect 
the Regard of other users and the Quality of items contributed by other users. Again, 
for the purposes of this simple example, it is assumed that only one item is 
contributed and rated at a time, although this would not necessarily be the case in a 
real situation. 

10 Fig. 4(a) shows that, once users have rated Item A in Fig. 3, the ratings for 

Item A affect the Regard of User 4, who contributed Item A. In the example, the new 
item contributed by User 4 receive high ratings by well Regarded users and User 4's 
Regard value goes up. In the example, User 4's Regard value rises from 0.5 (middle 
of the scale used here) to some higher value. 

15 Fig. 4(b) continues the example of Fig. 4(a), showing that User 4's changed 

Regard value affects the Regard and the item Quality of other users whose items User 
4 has rated in the past. In the example, User 4 has historically given low ratings to 
items contributed by User 3. When User 4's Regard rises, it negatively affects the 
Quality value for items contributed by User 3 that were previously rated by User 4. 

20 The rise in User 4's Regard also negatively affects the Regard of User 3 (not shown), 
since User 3's Regard is based on the ratings his items received from other users and 
on the Regard values of those users. In the example, User 4 has historically given 
high ratings to items contributed by User 1. When User 4's Regard rises, it positively 
affects the Quality value for items contributed by User 1 that were previously rated by 

25 User 4. The rise in User 4's Regard also positively affects the Regard of User 1 (not 
shown), since User 1's Regard is based on the ratings his items received from other 
users and the Regard values of those users. The example of Fig. 4(b) could be 
expanded to show more steps in which the newly affected Regard of Users 1 and 3 
affect the Regard of other users whose items Users 1 and 3 have rated, and the Quality 

30 of those items, and so on. 
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New ratings tor an item of one user can cause a cha^e in the Quality of that 
item. Similarly, new ratings for an item of one user can cause a change in the Regard 
of that user, which can potentially cause a change in the Regard of all users and in the 
Quality of items for all users. 

5 Figs. 5(a) and 5(b) show another simple example in a system having only four 

users showing that a user's Regard is affected by how others rate the items that the 
user contributes and by the Regard of those users and that a user's Regard can affect 
the Regard of other users and the Quality of items contributed by those other users. 

Fig. 5(a) shows that a user's Regard can be adversely affected as well. Again, 
10 it is assumed that the system has only four users and that only one item is contributed 
and rated at a time. In Fig. 5(a), User 1 contributes Item B, which is viewed and rated 
by Users 2, 3 and 4. In the example, User 1 was well Regarded when he contributed 
Item B. User 2 is poorly Regarded, having a Regard of 0.1. User 3 is also poorly 
Regarded, having a Regard of 0.1. User 4 is well Regarded, having a Regard of 0.9. 
1 5 In Fig. 5(a), poorly Regarded User 3 "loves" Item B, giving it a rating of 1 .0 (which in 
our example is the highest possible rating). Similarly, poorly Regarded User 2 
"loves" Item B, giving it a rating of 1.0. In contrast, well Regarded User 4 "hates" 
Item B, giving it a rating of 0.0. Even though two users love Item B, both of those 
users are poorly Regarded. 

20 Fig. 5(a) shows that, once the users have rated Item B, the rating for Item B 

adversely affects the Regard of User 1, who contributed Item B. In the example, the 
new item contributed by User 1 receives low ratings from well Regarded users (and 
below the average ratings for User l's previous items). Therefore, in the example, 
User l's Regard value falls to 0.5 (in the middle of the range) from some higher value. 

25 Fig. 5(b) continues the example of Fig. 5(a), showing that User l's changed 

Regard value affects the Regard of other users and the Quality of items contributed by 
other users whose items User 1 has rated in the past. In the example, User 1 has 
historically given low ratings to items contributed by User 2. When User 1 's Regard 
falls, it positively affects the Quality value for items contributed by User 2 that were 

30 previously rated by User 1, because his low ratings of these items are given less 
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weight. The fal 




ser l's Regard also positively affed 




ie Regard of User 2, since 



10 



15 



20 



25 



User 2's Regard is based on the ratings his items received from other users and the 
Regard values of those users. In the example, User 1 has historically given high 
ratings to items contributed by User 4. When User l's Regard falls, it negatively 
affects the Quality value for items of User 4 that were previously rated by User 1, 
because his high ratings of these items are given less weight. The fall in User l's 
Regard also negatively affects the Regard of User 4, since User 4's Regard is based on 
the ratings his items received from other users and the Regard values of those users. 
The example of Fig. 5(b) could be expanded to show more steps in which the newly 
affected Regard of Users 2 and 4 affect the Regard of other users whose items Users 2 
and 4 have rated, and the Quality of those items, and so on. 

The examples of Figs. 4 and 5 have involved systems that implement a Regard 
value for users. In contrast, Figs. 6(a) and 6(b) show a simple example in a system 
that implements an Expertise value for users. The example shows that a user's 
Expertise is affected by how others rate the items that the user contributes and that a 
user's Expertise can affect the Quality of others' items. Note however, that a change 
in the Expertise value of a user does not affect the Expertise values of other users. A 
user's Expertise value is changed when other users submit new ratings for that user's 
items. 

Fig. 6(a) shows an example in which User 4 contributes Item C, which is rated 
by Users 1, 2 and 3. User 2 has a high Expertise, but "hates" the item and give it a 
low rating. Users 1 and 3 both have low Expertise values, but "love" the item and 
give it a high rating. User 4's Expertise is based on the arithmetic mean of all ratings 
for all items contributed by User 4. Here, the new high ratings from Users 1 and 3 
have raised the mean of the ratings of User 4's items. Thus, the ratings from other 
users who rate User 4's items affect user 4's Expertise, but the Expertise of those 
other users does not affect User 4's Expertise. It is irrelevant that the users who love 
Item C both have low Expertise values. It should be noted that, in a system 
implementing Expertise instead of Regard, Quality is determined in much the same 
way as shown in Fig. 3 and as described below. That is, the basic Quality 
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determination fc 




item is made in accordance with 




Lgs received for the item, 



weighted by Expertise values of the users providing the ratings. 

Fig. 6(b) continues the example of Fig. 6(a), showing that User 4's changed 
Expertise value affects the item Quality of other users whose items User 4 has rated in 
5 the past. In the example, User 4 has historically given low ratings to items 
contributed by User 1 . When User 4's Expertise rises, it negatively affects the Quality 
value for items of User 1 that were previously rated by User 4. The rise in User 4's 
Expertise does not affect the Expertise of User 1. In the example, User 4 has 
historically given high ratings to items contributed by User 2. When User 4's 
10 Expertise rises, it positively affects the Quality value for items of User 2 that were 
previously rated by User 4. The rise in User 4's Expertise does not affect the 
Expertise of User 2. 

New ratings for an item of one user can cause a change in the Quality of that 
item. Similarly, new ratings for an item of one user can cause a change in the 
1 5 Expertise of that user, which can potentially cause a change in the Quality of items for 
all users. 

Again, it is emphasized that the above examples are simple examples, 
provided to aid in a basic understanding of the described embodiments of the present 
invention. A more precise discussion is provided below. The following paragraphs 
20 provide a discussion of terms and terminology used herein. 



The growth of the Internet and other wide area networks has multiplied the 
forms of and opportunities for, decentralized interactions. An item may be composed 
25 of words, whether ASCII text or text formatted by a word processing technology or 
hypertext mark-up language, text contained in a collection of data packets constituting 
an e-mail message, discussion group posting or Usenet newsgroup article, or the 
output of a process that translates from one language to another, or written words into 
spoken words, or spoken words into written words. An item may also be an 



2. Items 
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interactive, sequel 




exchange of words making up a^roup or one-to-one chat 




session. 



An item may be a fixed visual image, whether a drawing or image captured by 
a digital camera, or transferred from a photographic original to a scanned 
5 representation of the original. Streaming audio or video, whether live or previously 
recorded material and whether unidirectional, bidirectional or broadcast, would 
constitute an item, as would words, images, or streaming audio or video integrated 
into a web site. 



10 others, whether online or offline, may constitute an item. An example of this includes 
the course of a user's conduct in an online auction, whether as buyer or as seller. 
Another example is the performance offline of a subcontractor or of a general 
contractor, in their respective professional roles — whether assessed by each other or 
a third party — who entered into their agreement for the performance of services 

15 using an online medium. 

An item may also be information, goods and services, or assets which one user 
recommends to other users, or otherwise associates with oneself or one's reputation. 
An example of this is a recommended link to a third party website, or a link deep into 
the structured hierarchy of a website. Another example is an asset one puts forth for 
20 sale in an online auction. Another example is software, whether distributed as source 
code or in executable form and whether constituting a stand-alone program or 
operating system, or a replacement of or addendum to a portion thereof. An item may 
exist only as a pointer in records to locations accessible over, or data streams 
transmitted across, a wide area network. 



In various embodiments of the present invention, different users or types of 
users may have items associated with them and different users or types of users may 
participate in the calculation of Expertise, Regard, Quality and other measures 
included in the described embodiments of the present invention. 



More broadly, any single behavior or collection of behaviors viewable by 



25 



3. Users 
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ifl^odiments, users will overtly choose^^a 



In certain^Kbodiments, users will overtly choose x^Jarticipate. For example, 
a user might access a mailing list, discussion group, chat session or other items or 
grouping of items via a facility, such as a website, which is specifically enabled with 
features of the described embodiments of the invention. In some embodiments, this 
5 facility will be the only avenue to access the items or grouping of items and users 
would anticipate receiving in the ordinary course information regarding the Regard 
and Expertise (and related measures) of users and regarding the Quality (and related 
measures) of items. 

In other embodiments, some users need not ever access such a facility, or give 
10 any indication of interest in, or the intention of, assigning ratings to items contributed 
by other users, or give direct consent to the application of the measurements and 
methods of the described embodiments to themselves or to items they contribute. For 
example, a user might participate in a mailing list, discussion group, chat session or 
other grouping of items that is accessed by multiple facilities, only a subset of which 
15 are enabled with features of the described embodiments of the invention. More 
specifically, a user might choose to read and post Usenet newsgroup messages via a 
desktop application or website that has no support for features of the described 
embodiments, while other users access or contribute such postings via a facility that 
supports features of the described embodiments. 

20 In such embodiments, users who contribute items via facilities that are not 

enabled with any or all features of the described embodiments may also be 
incorporated into databases as item contributors and assigned Expertise or Regard 
values. The items contributed by such users may therefore be assigned Quality or 
related measurements and be subject to Vouching and Discrediting by other users and 

25 the other methods of the described embodiments, as discussed in various section 
below. 

4. Notation 

The following section discusses a preferred method of determining Quality 
30 and Regard from ratings supplied by users. User Expertise is also discussed. 
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i^BRnber of participating users, p grows flmc 



Let p be thMBnnber of participating users, p grows tWnclude each user added 
over time. 



Let i and j represent two users, where 

ij € {1,... ,p} 

Let c/ be the number of items contributed by user i. 



4.1 Ratings 



Let 

{0 if user i = user j 

A(v§) otherwise 
where n is an item contributed by user i, with 

ne {I, - ,Ci} 

10 

And ^3 is behavior by user j in response to item «, which behavior is mapped by the 
function H x ) to a floating point variable *j , with. 

0 < g% < 1. 

In alternative embodiments, 9 % can be any numerical scale for ratings, or any 
1 5 scheme that orders items to reflect user assessments or preferences. 

In alternative embodiments, behavior U S may include overt selection of a 
rating with a mouse or keyboard interface, a voice command, a movement of the hand 
or eye or other observable behavior by the human body. However, it is not necessary 
that ratings be overt, or that the user even be aware of participating in a ratings 

20 system. For example, behavior can also include the time spent viewing an item, the 
number of comments one interjects into a chat session, the decision to click on an ad 
banner, or a decision to bid on an auction opportunity. In the case of an item 
organized into a hierarchy, such as a web page with multiple levels of depth or 
specificity under subject categories, the behavior that triggers a rating may be the 

25 number of levels one traverses to lower levels. 
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In the eJ^Wihat user y does not connect with, englge in, or be associated with 
any behavior related to, the item, then 9i i = 0 . 

In certain preferred embodiments, such as those shown in Figs. 2(a)-2(c), the 
user interface can present sequentially a repeating series of highlighted or colored 
5 diamonds 242, or other alternating visual cues, each associated with a particular value 
of the rating variable. In a preferred embodiment, the rate at which the visual cues 
242 alternate is one second, although other intervals can be used. Upon any use of the 
keyboard or mouse, the currently colored or highlighted visual cue is interpreted as 
the user's rating ("single-click rating"). In order to limit ratings to items that a user 

10 has had a meaningful opportunity to review, in certain embodiments the visual cues 
and rating opportunity are not presented to the user until after the user has navigated 
to the end of the item and a fixed number of seconds has elapsed since delivery of, or 
the beginning of the user's interaction with, the item. In a preferred embodiment, the 
interval is 10 seconds. In certain other embodiments, use of keyboard or mouse 

1 5 commands to navigate to other items is interpreted as selecting the rating represented 
by the currently highlighted visual cue ("rate while you navigate"). Expressed 
differently, rating is an automatic or mandatory step in accessing items sequentially, 
or otherwise continuing to interact with the system, without exiting the system, the 
interaction or some level of the interaction. 



20 



Let 



Gi = 

(pxa) 



r 9ii 9 2 n 
9i2 **• 



L 9i P 



9% 



9% 



12 Cj 

We will also refer to the columns of G y as & > 9i i • • * > 9i • 



Let 



and 



R = 

(pxp) 



r n r 12 • • • rip 
r 2 \ ' ' - : 
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We will also refer to the rows of R as ri > r2 > 



» r p and the columns of R as 



r 1 , r 2 , 



4.2 Number of Ratings 



Let 



k n 



0 if user i = user j 

1 if user j has assigned a rating to the n th item contributed by user i 
0 if user j has not assigned a rating to the item 



10 



Let 



Ki = 

(pxa) 



Hi 

k 1 



k l 

L K ip 



*?1 



1 5 We will also refer to the columns of K( as k \^ k h - i k i 1 • 



Let 



so that 



20 



n=l 



0 < < Ci 



Let 



T = 



^11 *12 
_ tpl 



Hp 



U PP J 



We will also refer to the rows of Tas tl > t2 > and the columns of Tas 1 > 1 > • • > * P 
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5. Basic Expert Method 

The Basic Expert method determines K , the "Expertise" of a user, according 
to the ratings other users assign to items contributed by such user. 



In the initial embodiment of the method, Expertise is the arithmetic mean of 
the ratings that other users assign to items contributed by user i: 



In the initial embodiment, the Expertise of each user is calculated periodically, once 
10 every 12 hours. 

5.2 Default Expertise 

A user who has not contributed any items, or none of whose contributed items 
has been rated by any other user, is preferably granted the arithmetic average 
Expertise of users whose contributed items have received ratings: 



5.1 



Initial Embodiment 




otherwise 



J=ln=l 



15 



p 



i=l 



E { 



j=ln=l 

0 otherwise 



E s 



p 



j=ln-l 



0 otherwise 



Note that, in the initial embodiment, the first time any item contributed by a user 
receives a rating, the Expertise of the user is set equal to that rating. 
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6. Basic High Regard Method 

The Basic High Regard method determines K , the "Regard" of a user, 
according to the ratings that other users assign to items contributed by user / and the 
5 Regard of the users providing the ratings. 

6.1 Initial Embodiment 

Fig. 7(a) is an overview of data flows in an example High Regard ratings 
system, showing the circular nature of Regard affecting the Regard of other users and 
1 0 the Quality of items contributed by other users. 

As shown, one or more users contribute one or more items (also called 
content). A user does not have to contribute any item, but if he or she does contribute 
at least one item, the other users are given the opportunities to view and rate each item 
contributed. Each item is rated (for example, arrows 704, 709, 707) by one or more 
15 users. For example, as shown in Fig. 2(a), the user may rate a current message in an 
online forum. As another example, as shown in Fig. 2(c), the user may rate a web 
page. Various other types of items and groups of items are discussed herein. 

The ratings of the other users, along with the Regard of the users providing the 
ratings are used to create a Quality value of an item. Meanwhile, the Regard rating of 

20 the users giving the ratings are potentially being changed if those users are 
contributing items of their own or other users are rating their previously contributed 
items. As discussed below, not all embodiments determine Quality immediately when 
an item is entered. In a preferred embodiment, the Regard of users is updated 
periodically, for example, every twelve hours, although a shorter period of time can be 

25 used. In the future, as computing power increases, it is contemplated that both 
Quality and Regard will be updated more frequently. 

Fig. 7(b) is a more detailed example of Fig. 7(a). It will be understood that the 
"matrices" shown here are a conceptual presentation of a described embodiment of the 
present invention. An actual implementation of the matrices shown in Fig. 6(b) is 
30 discussed below in connection with the data structure of Fig. 11. The rating of each 
item 712 contributed by each user and rated by each of the other users can be stored in 
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;ord of ratings 718. The ratings 714 




to an item by the users, 



together with the Regard values 716 of the users, can be used to derive a measure of 
the Quality of the item 712. The Regard values of the users, are created by using the 
entire record of ratings 718. In some embodiments, an iterative method 714 can be 
5 used to simultaneously create the Regard values of the users and to use the Regard 
values to weight the ratings given by the users in the record of ratings. 

Once the Regard values 716 of the users are determined, one can then 
calculate the Quality 720 of an item as discussed below in Section 5 et seq. 



10 demand. Other embodiments calculate these values periodically and store them in a 
database for later retrieval. 

In the initial embodiment of the Basic High Regard method, Regard is the 
arithmetic mean of the ratings that other users assign to items contributed by user /, 



The Regard of each user is potentially an input into the calculation of the 
Regard of every other user, in addition to item ratings. Therefore, the High Regard 
method involves the simultaneous solution of a number of related equations. 

In the initial embodiment, the Regard of each user is calculated periodically, once 
20 every 12 hours. 

6.2 Default Regard 

A user who has not contributed any items, or none of whose contributed items 
has been rated by any other user, is granted the arithmetic average Regard of users 
25 whose contributed items have received ratings: 



Certain embodiments calculate user Regard and item Quality values per user 



( h l j otherwise 
weighting each rating by the Regard of the user providing the rating: 




15 
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Note that, in the initial embodiment, the first time any item contributed by a user 
receives a rating, the Regard of the user is set equal to that rating. 

5 6.3 Matrix Notation 



The interdependency of each user's Regard can be expressed in matrix 
notation. Setting: 





= Rh b = 


7*21 


7*12 " 
7*22 


* T lp " 




[Ml 

h b 2 


. w b p . 




- r pl 











a b 2 



= Th b = 



10 



tu 


^12 


* t\p 


hi 


*22 




tpl 









rMi 







then 



h b = 

(pxl) 



h\ 

h b 2 



4 
4 

h b d 



4 

h b d 



if a\ ± 0 
otherwise 

if a\ #0 
otherwise 



if a* * 0 
otherwise 



15 



7. Basic Quality Method 

The Basic Quality method determines , the "Quality" of an item, according 
to the ratings assigned to the item and the Expertise or Regard of the users providing 
the ratings. 



7 . 1 Initial Embodiment 



In the initial embodiment of the method, Quality is the arithmetic mean of the 
20 ratings that other users assign to item n contributed by user /, weighting each rating by 
the Expertise or Regard (the method being selected by the system operator) of the user 



Qd 



otherwise 
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providing the ratinj 



# 



10 



In the initial embodiment, Quality is calculated on demand — at the time it is needed 
for any purpose — preferably subject to, a time period following the most recent 
calculation, for example, a 10 minutes during which time a cached copy of the most 
recently calculated Quality value for the item is provided. 

7.2 Default Quality 

An item which has not received any ratings is granted the arithmetic average 
Quality of items that have received ratings. 

p 



P Ci 

EE { 

t=ln=l 



q? ifE*5*° 

J = l 

0 otherwise 



Qd = 



P Ci 

EE { 

i=ln=l 



i ifE*S^o 

0 otherwise 



Note that, in the initial embodiment, the first time an item receives a rating by any 
user, the Quality of the item is set equal to that rating. 



15 



7.3 Matrix Notation 

The calculation of Quality can be expressed in matrix notation: 



(lxl) 



hk? 







n n 
9il 


[hi h 2 ■■ 


■ hp] 








. 9i P . 






K il 


[hi h 2 


■ k) 








L K ip J 



if E*B # 0 

3& 



otherwise 



8. Analysis and Illustration 
8 . 1 Basic Expert Method 

20 An assumption of the method is that an item rating is an effective 

measurement of the capacity of the user who contributed the item to identify valuable 
items contributed by other users. From the point of view of the user assigning item 
ratings, the user is helping to identify experts whose assessment of the value of items 
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# # 

will receive highe^weighting. From the contributor's poinWJF view, high item ratings 
mean higher Expertise, implying more influence over Quality levels. In this sense, the 
Basic Expert method is an expert system. 

Expertise under the Initial Embodiment 

In a universe of four users i 9 j 9 x and z, with user i having contributed at least 
one item and users j, x and z having each rated at least one user i item: 

- _ Tij + r ix + r iz 



t{j 4* ti x ~f~ ^tj 



In the calculation of Expertise, item ratings are weighted equally, Regardless of which 
users assigned the ratings. 

10 Partial differential equations demonstrate the impact on user z's Expertise of an 
additional item rating by user j: 

rating/Expertise differential 

dh _ g?j- Jk 



tij H- tix ~r* 
total # of previous item ratings 



f*>o«fiB>*. 



User z's Expertise increases whenever another user rates an item contributed by user i 
15 above user fs previous level of Expertise. The increase is equal to the difference 
between the rating and the previous level of Expertise, divided by the total number of 
ratings previously assigned to user i items. 

Quality 

If users x and z have already assigned ratings to item n contributed by user i: 

-n _ K9& + h z g? 2 

Hi 



h x + h z 



The impact on item Quality of a rating by user j is: 

rating/Quality weighting by 
diffe rential r atel " s Expertise 

dUj h x + h Zj 

total Expertise of 
previous raters 



601 95430v 1 



7Q 





The Quality of item n contributed by user / increases whenever another user rates the 
item above its previous level of Quality. The increase is equal to the difference 
5 between the rating and the item's previous level of Quality, weighted by the Expertise 
of the user providing the rating, divided by the total Expertise of users who previously 



rated the item. 

8.2 Basic High Regard Method 

An assumption of the method is that an item rating is a useful measurement of 
10 the capacity of the user who contributed the item to identify talented item 
contributors. 

From the point of view of the user assigning item ratings, the user is helping to 
identify experts whose assessment of the value of other users' items will further refine 
the selection of experts. From the contributor's point of view, high item ratings mean 
15 higher Regard, implying more influence over item Quality and more influence over 
the Regard of other users. In this sense, the Basic High Regard method is a robust 
expert system. 

Regard under the Initial Embodiment 

Again in a universe of four users, with user i having contributed at least one 
20 item and users j 9 x and z having rated at least one item contributed by user i: 



In the calculation of Regard, the sum of all the ratings that each user has assigned to 
items contributed by user / is weighted by the rating user's Regard. 

The impact on user z's Regard of an additional item rating by user j is: 




rah) + r ix h b x + r iz h b z 
Ujh) + t ix h b x + t iz h b 



weighting by 



rating/Regard rater » s Regard cross-rating dependencies 



dh\ 




total weighted # of previous item ratings 
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User z's Regard increases whenever another user rates an item contributed by 
user i above user z's previous level of Regard. 

The increase is equal to the difference between the rating and user z's previous level of 
5 Regard, weighted and adjusted, divided by the total weighted number of ratings 
previously assigned to user z items. 

§|>0if g%><ff 

The weighting in the numerator is the rating user's Regard. The adjustment for cross- 
rating dependencies accounts for the effect of the additional rating on the Regard of 
users other than i — and hence, how the rating affects the weight assigned to their 
10 previously assigned ratings of user z items. Cross-rating dependencies can magnify or 
diminish 
ahi 

dt a , but cannot change its sign. 



Quality 

If users x and z have already assigned ratings to item n contributed by user i: 



15 



Qi hi + h\ 



The impact on item Quality of a rating by user j is: 



weighting: 

rating/Quality rater's Regard cross-rating dependencies 

differential 



W-tf)(«j + ffi + M-tf)gl[ + M-tf)gS 

dttj h b x + h" z 

total Regard of previous raters 



ft >o if fl § > h\ 

The impact of a rating on item Quality is similar in form to the impact of a rating on 
the contributor's Regard. The Quality of item n contributed by user i increases 
20 whenever another user rates item n above the item's previous level of Quality. 

The increase is equal to the difference between the rating and the item's previous level 
of Quality, weighted and adjusted, divided by the total Regard of the users who 
previously rated the item. 

The weighting in the numerator is the rating user's Regard. The adjustment for cross- 
25 rating dependencies accounts for the effect of the additional rating on the Regard of 
users other than z — and hence, how the rating affects the weight given their 
previously assigned ratings of item n. Cross-rating dependencies can magnify or 

diminish , 
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sign. 



8.3 Rating Dependencies 
Basic Expert Method 

An item rating can affect the Expertise of the user who contributed the item, but 
not the Expertise of any other user. However, a rating of an item contributed by one 
user can affect the Quality of an item contributed by another user. A simple example 
of this is discussed above in connection with Fig. 6. 



• user j rates item n: 

- above item n's Quality — ► Quality rises 

- above user i's Expertise — > Expertise rises 



• user i's previous rating of item m: 

- above item m's Quality — > Quality rises 

- above user x's Expertise — » Expertise unchanged 



Table 1. Basic Expertise: Rating Dependencies 



In the example case, a rating of item n contributed by user / can affect the Quality of 
item m contributed by user x (already assigned ratings by all users): 



iHi (n) 
I 

x (m) 



rating dependencies 



3«? 




dtij 



hi -h hj + h z 



total Expertise of users 



where 



dhj 



tij H - t{ x -j- t{ t 



'XZ 



§f£ >0if >^and 5 ^>^ 
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When user j ratdmm n above user f s previous level oHSpertise, user z's Expertise 
rises. If user i has already rated an item m above its previous level of Quality, then 
item m's Quality rises. 

Thus, an item rating potentially affects the Quality of the item, the Expertise of 
the contributor and the Quality of every item ever rated by the contributor. A rating 
has secondary effects that extend as far as one other user and that user's rating 
activities. The method recognizes an item rating as, in part, an indication of the 
ability of the contributor of the item to identify valuable items contributed by other 
users. 



1 0 Basic High Regard Method 

An important distinction of the Basic High Regard method is that one user's 
rating of an item contributed by a second user can affect the Regard of a third user. 



j r~H ^ (n) 

I 

x (m) 



user j rates item n: 

- above item n's Quality 

- above user z's Regard - 



Quality rises 
Regard rises 



• user i 9 s previous rating of item m: 

- above item ra's Quality — > Quality rises 

- above user re's Regard — > Regard rises 



Table 2: Basic High Regard: Rating Dependencies, Sparsely Rated Items 

15 

Thus, the method recognizes an item rating as, in part, an indication of the ability of 
the contributor of the item to identify other talented items contributors. The rating of 
an item by a contributing user affects the Regard of other users in the system. 



601 95430v 1 



3 (p)[zEI* ( n ) 

IT & it 

z (o)t=5x (m) 

• user j rates item n: 

- above item n's Quality Quality rises 

- above user i's Regard — ► Regard rises 



• if all the users have rated each other's listed 
items, the Regard of each user and the Quality 
of each item may change. 



• as the Regard of users change, their previous rat- 
ings of user j and item n may receive greater or 
lesser weight, circling back to magnify or dimin- 
ish, but will not entirely offset, the impact of user 
fa rating. 




Table 3: Basic High Regard: Rating Dependencies, Fully Populated 
An item rating has ripple effects that can propagate through the entire network, 
potentially affecting the Regard of the user who contributed the item, the Regard of 
5 the user who assigns the rating, the Regard of other users, the Quality of the item and 
the Quality of other items. 

A useful example is a situation in which cross-rating dependencies are limited 
to two parties. For example, if users j and z have previously assigned item ratings that 
are equal to the Regard values of the contributors, cross-rating dependencies are 
10 limited to users i and jc. 

In this case, the formulas for the impact of an additional rating by user j on 
user i's Regard reduce to: 

cross-rating dependency 

rating/Regard weighting by 

between users 1 and x 
di fferential rater's Regard , * s 

dUj tijhj 4- t ix h x + t iz h b z 

* v ' 

total weighted # of user i item ratings 

cross-rating dependency 
between users i and x 

' A 

dh b 

dh b x {Txi " ^dtTj 



dUj t xi h\ + t xj h° + t xz h b 
- v ' 

total weighted # of user x item ratings 
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14 



10 



\ iSroove user Vs previous level of RegarcCTRer 



Thus, if the rating Wroove user Vs previous level of Regard^Ren 

9$ ~ h\ > 0 

and, because cross-rating dependencies can diminish, but not reverse, the impact of a 
rating 

dtij 
dtij 

If user i has rated items contributed by user x on average higher than user x's Regard, 
then 

r xi - h\t xi > 0 

and 

If user x has rated items contributed by user i on average higher than user i's previous 
level of Regard, then 



r ix - Kt ix > 0 



15 and 



Hence, the impact of the rating is magnified; i.e., it exceeds the rating/Regard 
20 differential weighted by user fs Regard. 

The alternatives can be presented in tabular form: 
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^ Magnified or 
Diminished 



3 ED i 

it } Cross-Rating Dependency 

x 



Average user i rating of user x items 



Average user x ratings of user i items 


above Regard 

(r xi - h b x t xi > 0) 


below Regard 

(r xi - h b x t xi < 0) 


above Regard (r ix - h?t ix > 0) 


Magnified 


Diminished 


below Regard {r ix — h\t ix < 0) 


Diminished 


Magnified 



Table 4: Impact of Cross-Rating Dependencies on Regard (Outsider Rating) 

This result also applies to larger numbers of users with a populated data set. 
5 When users owe their Regard, in part, to cross-rating each other — or have mutually 
reduced each other's Regard — their Regard is subject to greater volatility under some 
circumstances. A rating by a user without a cross-rating dependency, such as user j in 
the example, will have greater impact than suggested by the user's Regard alone. 

10 A similar result holds when one user rates an item contributed by another user with 
whom the user shares a cross-ratings dependency. 



dt x i 

Magnified or i 

Diminished { [XJr } Cross-Rating Dependency 
x (m) 



Current (item m) + Average user i 
ratings of user x items 



Average user x ratings of user i items 


above Regard 

(9%+r xi -(l + h b x t xi ) >0) 


below Regard 

(9%r xi -{l + h x t xi ) <0) 


above Regard (r^ — h\ti x > 0) 


Magnified 


Diminished 


below Regard (r» x — h\u x < 0) 


Diminished 


Magnified 



Table 5: Impact of Cross-Rating Dependencies on Regard (Cross-Rating Party Rating) 

15 

This aspect of the High Regard method facilitates construction of user 
"communities". Users who associate and build a web of positive cross-ratings can 
help each other increase their Regard quickly — the effect of their positive ratings on 
each other's Regard is magnified. 

20 
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This asp^^f the described method also heighteiiHRe risk of mutual negative 
ratings, in the course of a form of destructive behavior often observed on wide area 
networks, known popularly as "flaming" — intentional provocation or repetitive 
attacks, typically in the form of an exchange of written messages. Two or more users 
5 engaging in mutual attacks lose Regard quickly — the effect of their negative ratings 
on each other's Regard is magnified. 

Ratings assigned by "outsiders", users outside the web of cross-ratings, can 
have similar effects. An outsider who boosts the items of members of a community 
10 can prompt a magnified increase in their Regard. An outsider who rates negatively 
the items of users engaging in flaming can cause a magnified drop-off in their Regard. 
However, these magnifying effects do not circle back to affect the outsider's Regard, 
because there are no cross-rating dependencies as to the outsider. 

15 Users who associate in communities or engage in flaming, who subsequently 

reverse course and rate each other in the opposite direction, have a diminished impact 
on Regard. In fact, any time that two users or groups of users rate each other's items 
in opposing directions, it creates conflicting cross-rating dependencies. In effect, a 
user is either weakening one of the sources of the user's own Regard, or strengthening 

20 a force that is acting against the user's own Regard. 



25 



^ Magnified or 
Diminished 

j — > i -» x 

\ i . 

z 



Table 5: Chain of Dependencies 

30 Similar overall effects can arise when, instead of direct cross-ratings, there is a chain 
of dependencies. Whether the original rating is magnified or diminished is a question 
of the product of the signs of these links in the chain. For example, if user i has rated 
items contributed by user x above user x's Regard (+), user x has rated items 
contributed 

35 by user z below user z's Regard (-) and user z has rated items contributed by user i 
below user z's Regard (-), then the product of the signs is 

(+) x (-) x (-) = (+) 

In this case, user y's rating of user / is magnified. 
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^7 



atij 



10 



Cross-Rating Dependencies — Quality 

Deriving Quality in the same two-user cross-ratings scenario triggers the same 
cross-ratings effects. 

The formulas for the impact of a rating by user j of item n contributed by user i reduce 
to: 

cross-rating dependency 
rating/Quality by ^ . ^ % 
differential ratertRegard , 

dtij h b x + h b z 

total Regard of previous raters 



15 



20 



cross-rating dependency 
between users i and x 

dh b x _ [rxt nM dUj 

dtij t xi h\ + t xj h) + t xz h b z 
y ^ ' 

total weighted # of user x item ratings 



Thus, if the rating is above item w's previous level of Quality, then 

9-j - Q? > 0 



and, because cross-rating dependencies can diminish, but cannot reverse, the impact 
of a rating 

dtij 

25 If user i has rated items contributed by user x on average higher than user x J s Regard, 
then 



and 



> 0 
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If user x has rated item « contributed by user i above its previous level of Quality, then 
and 

5 

Hence, the impact of the rating is magnified; i.e., it exceeds the rating/Quality 
differential weighted by user f s Regard. 

10 The alternatives can be presented in tabular form: 



|^ Magnified or 
Diminished 

j [3 z(n) 

it } Cross-Rating Dependency 

x 



Average user i rating of user x items 



User x rating of user z's item n 


above Regard 

(r xi - h\t xi > 0) 


below Regard 

(r xi - ti?t xi < 0) 


above Quality - q? > 0) 


Magnified 


Diminished 


below Quality (g? x - q? < 0) 


Diminished 


Magnified 



15 



Table 7: Impact of Cross-Rating Dependencies on Quality 



9. Extended Methods 



9. 1 Expert Methods 



20 



A general formulation for alternative embodiments of the Expert method 
(the "Extended Expert methods") is: 



P *i 
j = ln = l J 



otherwise 
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9.2 



gard Methods 



A general formulation for alternative embodiments of the High Regard 
Method (the "Extended High Regard methods") is: 



j = ln=l 



otherwise 



9.3 Quality Methods 



A general formulation for alternative embodiments of the Quality method (the 
"Extended Quality methods") is: 



if £*S#o 

3=1 

otherwise 



10 



9.4 Transformation Fimction 



®( x ) is a transformation function and ^( x ) is a function that reverses the effect of 



15 Note that if 



and 



*(*(x)) — x 



*(x) = X 



20 



25 



then the Extended methods calculate arithmetic means — weighted, in the case of 
Regard and Quality. In this case, the Extended Expert method is equivalent to the 
Basic Expert method, the Extended High Regard method is equivalent to the Basic 
High Regard method and the Extended Quality method is equivalent to the Quality 
method. 



If 



$(x) = log(x) 
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and 



15 



then the Extended methods calculate geometric means. 

= exp(x) 



If 



and 



*(*) = " 

x 



¥(*) = - 

X 



1 0 then the Extended methods calculate harmonic means. 



If 



and 



= x n 



*(ar) = x n 



where n is a parameter set by the system operator, then the Extended methods 
calculate Holder means. In alternative embodiments, n may be set to any real number. 
In one embodiment, n=3. In another embodiment, «=l/3rd. 



20 



9.5 Default Value 

In alternative embodiments, the Expertise or Regard granted a user who has not 
contributed any items may be determined in accordance with: 



*5 = * 



i—l 



j=ln=l 

0 otherwise 



P Ci 



j-ln=l 

0 otherwise 



25 



in which h t represents the Expertise or Regard, as the case may be, of user /"; or, more 
generally, an amount otherwise related to the ratings assigned to items contributed by 
other users: 
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or simply set to zero: 

h e d = 0 

The Quality granted an item to which no user has assigned a rating may be determined 
5 in accordance with: 



p «i 
E E < 


J = l 

0 otherwise 

v. 


t E < 

i=ln=l 


i if E*5^° 

0 otherwise 



or, more generally, an amount otherwise related to the ratings assigned to other items: 

q d = e(g}...gf\... 

or simply set to zero: 

q<t = 0 

10 

9.6 Periodicity 

In alternative embodiments, the Expertise or Regard of each user, as the case may be, 
is calculated periodically, after the passage of a specific period of time, after the 

1 5 collection of a specific number of new item ratings, or as often as possible given the 
available computational resources. In alternative embodiments, the Quality of each 
item is calculated every time the item's Quality is requested or required, subject to a 
minimum time period following the most recent calculation, during which time a 
cached copy of the Quality value most recently calculated is provided; or after the 

20 passage of a specific period of time, after the collection of a specific number of new 
item ratings, or as often as possible given the available computational resources. 

9.7 Matrix Notation 

The interdependency of each user's Regard in the Extended High Regard 
25 method can be expressed in matrix notation. 

Let 
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Setting: 



n=l 



and 



R* = 



r 



r 21 



12 



'22 



IP 



PP J 



' 11 



'21 



'pi 



12 



'22 



PP J 



/i| 



10 



then 



"of" 


= Th e 







/If 



p J 



*n ti2 

^21 ^22 
_ t p i 



hp 



ifaf#0 
/i£ otherwise 

M otherwise 



ifa^O 



h% otherwise 



The calculation of Quality can also be expressed in matrix notation: 



15 



(lxl) 
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[/if h\ ■■■ h;] 



[hi 



43 



K il 
K i2 



if £a§ * 0 



10. Computar^ial Method 
10.1 Description 



The following procedure solves for Expertise M in one step and for Regard h t by an 
iterative method of successive approximation. 

10.2 First Step 
Let 



h\ 
K 



P 



if E*u ± 0 



K hd otherwise 



£*2j 

j=l J 



if £*2j 7^0 

i=l 



h d otherwise 



if E*pj #o 



otherwise 



10 where is a selected transformation function: 



'5 = 

n=l 



15 



and is determined according to the selected procedure for calculating a default 
value. In one embodiment, in this first step: 



i=l J = l 



In each case, using a function ^( x ) , for which: 

Stopping here results in M , Expertise under the Extended Expert method; and if: 

$(x) = x 

then h i is equivalent to h i , Expertise under the Basic Expert method. 
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AA 



10.3 



Second Step 



Calculation of Regard requires additional steps. 

A first approximation of each h t value is calculated as follows: 



w i 

Wo 



= R*h e = 



r r * 



r 21 



12 



'22 



r lp 



pp 









h\ 




-k- 



= Th e = 



hf 
h% 1 



*11 *12 



Hp 



v pp J 









H 




-k- 



*(%-) if a? 1 * 0 



hj l otherwise 

*(f) 



if a£' ^ 0 



otherwise 



otherwise 



if ai* jL 0 



where ft i is the Expertise of each user calculated above and h * 1 is determined 
10 according to a selected procedure for calculating a default value. In one embodiment, 
in this second 

step, the value is based on the M values determined in the first step. 



0 otherwise 



t 



i if jx- ^ o 

0 otherwise 



Using a function ^( x ) , for which: 



15 
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10.4 Subslfuent Steps 



The following procedure is repeated successively with /w=(2,3,4, ... ), converging 
toward a solution. 





= R*h em - 1 


r r* 
"ll 

r 21 


"12 
r 22 


r ip 




■ ti^- 1 " 






L r P i 




'pp j 







hi tn 

*22 

t P i ••• 



Hp 



"PP J 



hp- 1 



hj" otherwise 
/ij" otherwise 



h e d m otherwise 



10 where h d" is determined according to a selected procedure for calculating a default 

value. In one embodiment, in this second step, the value is based on the h * values 

determined 

in the previous step. 



h e d m = * 



f p 

C m - 1 



P 



W"- 1 ) if£^o 

0 otherwise 



1 if£to#o 

0 otherwise 



15 



Using a function *( a: ) , for which: 



and 

lim = h\ 

m— ►oo 
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The procedure converges toward h t , Regard under the Extended High Regard method 
and if 

<b{x) = x 

5 then h i is equivalent to h \ , Regard under the Basic High Regard method. 
Convergence 

This procedure can be applied successively a specified number of times m. In one 
embodiment, m=5. In other embodiments, the procedure can be applied successively 
until the approximation of Regard at the beginning and at the end are nearly equaL In 
10 one embodiment, the procedure is considered complete when: 

max|?(A^ - /i- m_1 )| < -005 

1 1 . Inertia 

11.1 Expertise/Regard Inertia 

1 5 Under the Expert and High Regard methods, the first time that any of the items 

contributed by a user receives a rating, the user's Expertise or Regard, as the case may 
be, is set equal to that rating. This could have undesirable consequences. The items 
contributed by a user may be rated initially by a poorly Regarded user and for a period 
of time thereafter, by a small or poorly Regarded subset of users. 

20 In order to control the amount by which item ratings influence the Expertise or 

Regard of relatively new or infrequent users, we create a notional user /=1, with 

r ix = a{t\t u r\Ti,h e ) 

and 

25 where h e denotes the vector of Extended Expertise or Regard values, as the case may 
be, for all users. 

Thus, the notional user assigns r n and Ui for each user, taking into account 
the number and level of ratings assigned to user f s items by other users, the number 
and level of ratings user i has assigned to items contributed by other users and the 
30 Expertise or Regard of all users. 

In certain embodiments, T %\ is set equal to tilh d and Ui is determined 
according to an equation in the form of: 

a — (bx -f cy — y/dxy) e 
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where a,b 9 c,d and e are constants selected by the system operator and x represents the 
aggregate number of the user's items that have been rated by other users and y the 
number of other users' items that have been rated by the user. Using a relatively high 
5 value for a and small (positive) values for b,c 9 d and e, this formulation has the 
characteristic of reducing the fixed constant a more slowly when x and y grow in 
tandem and more quickly when x andy grow divergently. 

This property causes the system to withdraw inertia as a user interacts with the 
system, but at different rates: more slowly from a user who rates items and whose 
10 items are rated by other users close to a targeted ratio and more quickly from a user 
who primarily does one but not the other. 



Fig. 8 is a three-dimensional plot of a High Regard Inertial Model having two 
axes labeled from 0 to 100 representing, along the first axis, the number of ratings 
other users have assigned to items contributed by the user in question and, along the 
second axis, the number of ratings the user in question has assigned to items 

20 contributed by other users. A third time axis labeled from 0 to 500 represents the 
number of notional ratings (set at a predetermined default level of Regard) assigned to 
an item contributed by the user in question in order to create the inertia effect. As can 
be seen, the effect of inertia diminishes as other users assign ratings to items 
contributed by the user in question and as the user in question assigns ratings to items 

25 contributed by other users. The shape of the three-dimensional plot represents the 
characteristic of the specified embodiment that the inertia effect diminishes more 
slowly when the user in question rates items contributed by other users and the user in 
question's contributed items are rated by other users, in tandem. The described 
embodiment reflects a target ratio. 

30 The Regard inertia effect in the described embodiment diminishes rapidly, in 

contrast, when the user in question rates items contributed by other user often, but few 
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Example 



r 5 i = hihZ 




\ n =2 n=2 \ n=2 n~2 



users rate items comffbuted by the user in question, or vic^rersa. In the described 
embodiment, the number of notional ratings is initially 500. 



1 1 .2 Item Quality (Quality inertia) 

5 Under the Expert and High Regard methods, the first time that any item 

receives a rating, the item's Quality is set equal to that rating. This could have 
undesirable consequences. The item may be rated initially by a poorly Regarded user 
and for a period of time thereafter, by a small or poorly Regarded subset of users. 

In order to control the amount by which early item ratings influence the Quality of 
10 items, we create a notional user /=2, with 

and 

where d represents the number of units of time, using a standard unit selected for a 
preferred embodiment (e.g. minutes, hours or days), measuring the time elapsed since 
1 5 user i contributed item n. In the following example, d is expressed in minutes. 

In other words, the notional user assigns a 9& and *« for each item, taking into 
account the Quality of other items with a track record of ratings, time elapsed since 
the item was contributed and the number and level of ratings assigned to the item by 
other users. 

20 In certain embodiments, 9& is set equal to *«9S , with the formula determining *S in 
the form of: 

a — (bx + cy — y/dxy) e 

In this case, x represents the aggregate number of ratings assigned to the item and y 
the time interval d. This formulation causes the system to withdraw inertia as users 
25 rate the item and with the passage of time, but at different rates: more slowly for an 
item that receives ratings at a targeted rate per period of time and more quickly for an 
item that receives ratings at a higher rate (resulting in adequate data) or at a lower rate 
(suggesting disinterest). 

Example 

9?j = *£95 



= max{l, 600 

30 
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Fig. 9(a) 




three-dimensional plot of a Quality 




;ia Model having a first 



axis labeled from 0 to 300 units of time as specified above, a second axis labeled from 
0 to 80 representing the number of ratings users have assigned to the item in question 
and a third time axis labeled from 0 to 600 representing the number of notional ratings 
(initially set to a predefined default level of Quality) assigned to the item in question 
in order to create the inertia effect. As can be seen, the effect of a rating diminishes 
with the passage of time and as other users assign ratings to the item in question. The 
shape of the three-dimensional plot represents the characteristic of the described 
embodiment that the inertia effect diminishes more slowly when users rate the item in 
question steadily in small numbers. The described embodiment reflects a target 
number of ratings per units of time. The Regard inertia effect diminishes rapidly, by 
contrast, when considerable time passes with very few ratings contributes, or if a large 
number of users rate the item during a short time interval. 

11.3 Clarification 

Notional user values created for calculations of Quality will, in some 
embodiments, be excluded from the calculation of High Regard values. 

12. Decay 

12.1 Description 

In order to keep Regard and Expertise values current, or to discount older or 
unusual historical results, some embodiments will include "Decay", procedures for 
adjusting the record of ratings with the passage of time. 

12.2 Simple Time Decay 

In one embodiment, a standard time period, e.g. days, weeks or months, 
determines the rate at which ratings are removed from the calculation. 

Let g be the number of such standard time periods from which ratings will be included 
in the calculation of Expertise or Regard, as the case may be, after which they are 
removed. 

Let f m and R m be the (p x p) matrices of *tf and r ^ values reflecting only the 
ratings provided during a single period m (Regardless of the date the item was 

contributed). T m and R™ , defined below, substitute for T and R to calculate Expertise 
or Regard, whichever is used in the embodiment. 

For m=\, let 



T = T 1 



'171 
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and 



For 2 < ™ < 0 




and 

For m > 9 
and 

12.3 Segmented Decay 

Other embodiments of Decay treat differently subgroups of users segmented 
according to one or more user characteristics. In some embodiments, it is helpful to 
divide the users into a number of subsets and to use the method of "segmented decay" 
to alter the number and value of ratings among the record of ratings for items 
contributed by particular groups of users, applying systematic but different procedures 
to each subset. Users can be divided into subsets according to their Regard (or 
Expertise) values, or some other appropriate attribute. For example, one can make the 
ratings given to items contributed by the most poorly Regarded users "sticky," that is 
largely unaffected by decay procedures, or even strengthen the worst ratings under 
some circumstances. In this way, users with the lowest level of Regard who are prone 
to disruptive behavior are largely and permanently excluded from user interactions. 
Alternatively, one can make the worst ratings given to items contributed by both well 
Regarded and poorly Regarded users decay faster over time than for users who have 
more average Regard values. In this way, well Regarded users have an incentive to 
make continuous contributions to stay highly ranked and poorly Regarded users will 
receive encouragement and assistance to improve their Regard. Additionally, one can 
use Decay to encourage users to contribute ratings of items contributed by other users, 
for example, by causing the better ratings of items contributed by well Regarded users 



601 95430v 1 



to persist longer, IKuch users actively rate items contriWroa by other users. The 
following paragraphs describe an embodiment of segmented decay, as expressed 
mathematically. 



5 Let 0 ( h ' u ) be a function that maps users into subsets u \ » ■ • • > U z . In some 
embodiments, this function will segment users according to their previously computed 
Expertise or Regard. 

A standard time period, e.g. days, weeks or months, determines the rate at which 
ratings are affected by a transformation function using the procedure set forth below. 

e x e x T R 

10 Let T m and R m be 0>*p) and <p*p) matrices including only ratings provided during 

period m (by any user in any subset) for items contributed (at any time) by users in 
subset 0x . 

To compute either Expertise or Regard with segmented Decay, substitute: 

1=1 

and 



r — y ^ Rm 

x=l 

15 

using T™. and defined below. 



A transformation function S is selected, such that 
and 

Example 

Ox ™ e * e * 

R rn = Y,S R {l,™,x^R l ,T l ) 

i=i 

User i e C/f if 0 < h { < 0.25 
20 In one embodiment, #( ft > U) separates users according to four levels of Expertise or 
Regard 



User i € U$ if 0.25 < hi < 0.50 
User i € Ul if 0.50 <h { < 0.75 
User i € U\ if 0.75 < hi < 1.00 
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The procedure adjWis the record of ratings as follows. WrWia user provides a rating 
of an item contributed by another user, the rating remains in the record unaffected 
during the following 10 standard time periods. All such ratings are removed from the 
record after 25 

e x e x 

5 standard time periods. Between the 10 th and the 25 th period, T and R are 

adjusted (according to different formulas for each subgroup) which lessen the impact 
of each rating as the periods progress and adjust older ratings from lower values to 
higher values in the case of poorly Regarded users and from higher values to lower 
values in the case of well Regarded users. 

10 The procedure continues to base Expertise or Regard, as the case may be, on 

item ratings assigned over an extended period of time, but lightens the weight of bad 
historical ratings on poorly Regarded users and reduces the benefit of good historical 
ratings for highly Regarded users. These effects help users with poor ratings to 
recover Expertise or Regard and encourage users with high ratings to make additional 

15 contributions. 

For all subsets e * € #2, #3> 0a} 9 when m=l, let 

e x e x 

and 



For 2 < m < 10 



20 and 



Rtti — R n 



e x o x e x 
T m = T 1 + h T" 1 



9 X *« 0^ 



For 11 < rn < 25 



e x e x 8* 9 =r J* 

Tm = T™ + - ■ • + T" 1 " 10 + p^T™- 11 + • • • + pZJ 1 

ifm>\2 



and 




For > 26 



25 and 



T m — 7™ -\ h + p^T™" 11 + • • • + pl 5 T m ' 25 



= R m H h A™" 10 + p^T™" 11 + • • • + P^T™- 25 
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# 



The following formula reduces proportionately the impact of ratings over the Decay 
period: 

for all 9x 



% 26 - Z 



Pi = 



15 



The following formulas cause ratings of each subgroup to move closer to the mean 
over the Decay period 



0i 0i 

p? = pT 



i /4irlV 8)( ' _,1) 
4 V"i5rv 



4 V&TJ 



0 2 02 

= pT 



2 ^"^J 



1 /2r^\ 



,>. (.85)«-"> 



tf3 0 3 

p? = pT 



2 V^rr/ 



/ v (.85)('- ll > 



2 \ / 



_, X (.85)('- U ) 



04 



04 

T 



(,8)(i-n) 



^^v- 8)(i_ll> 



Fig. 9(b)) is a graph of a Segmented Decay. In accordance with the mathematical 
10 model, Fig. 9(b) displays the different adjustments made to ratings assigned to items 
contributed by users in four different categories of Regard. On a scale from zero to 
one, the first group 0, represents users with Regard of greater than 0.75 to 1.0 
(inclusive). The second group 9 2 represents users with Regard of greater than 0.50 to 
0.75. The third group 6 3 represents users with Regard of greater than 0.25 to 0.50. 
15 The fourth group represents users with Regard from 0.0 to 0.25. The horizontal axis 
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represents time p^rods 11-25 following the contribution^^ any rating. The time 
periods can represent days, weeks, or months, or some other time intervals. Time 
periods 0 through 10 are not represented on the graph because in the described 
embodiment, segmented decay does not begin to affect any rating during the first ten 
5 periods following the contribution of the rating. The described embodiment causes 
ratings to regress toward the rating of 0.5, represented by the movement of the 
diamonds, triangles, circles and spades toward the mean given the passage of time. 

Expressed mathematically, this is a transformation of elements of the R 
matrix. The solid straight line represents the incremental reduction in the weight 
10 assigned to older ratings. Expressed mathematically, this is a transformation of 
element of the T matrix. 



13. Vouch/Discredit 
13.1 Description 

15 In the case of items that have recently entered the system and therefore 

received few ratings, a robust calculation of Quality will not be available. In this early 
period, the Expertise or Regard (whichever is used in the embodiment) associated 
with the user who contributed the item may be the primary indicator employed by 
other users to predict whether the item has value and is worth review. 

20 "Vouching" permits users to associate their Expertise or Regard, as the case 

may be, with an item contributed by another user, as if they had contributed the item 
themselves. The procedure is designed to help solve the sparse ratings problem by 
giving users with relatively high Expertise or Regard an incentive to identify good 
items contributed by users with relatively low Expertise or Regard and to use their 

25 Expertise or Regard to bring the item quickly to the attention of the community of 
users. The Vouching user's higher Expertise or Regard value is thereafter associated 
with the item, instead of the Expertise or Regard of the user who contributed the item, 
in procedures that filter, highlight, sort or otherwise evaluate items based on the 
Expertise or Regard of the contributor. As the item subsequently receives ratings, the 

30 user who Vouched for the item receives credit — or potentially loses Expertise or 
Regard — to some extent, as does the user who contributed the item. 

"Discrediting" does the opposite: it permits users to dedicate their Expertise or 
Regard, as the case may be, to impugn an item submitted by another user. It is 
designed to give users an incentive to identify bad items contributed by users with 
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relatively high Ex^^e or Regard, quickly alerting the coi^feiity of users. A value 
equal to one minus the Expertise or Regard of the user who Discredits the item is 
thereafter associated with the item, instead of the Expertise or Regard of the user who 
contributed the item. As the item subsequently receives ratings, the user who 
5 Discredited the item receives credit — or potentially loses Expertise or Regard— but in 
an inverse relationship to the ratings received by the item. 

13.2 Initial Embodiment 

In the initial embodiment, when user j Vouches for the rfl 1 item contributed by 
10 user z, new columns are added to the K i and matrices, such that: 

k?+ l = A? 

and 



15 where indicates the establishment of a link with column n of the and Gi 
matrices. The newly created (and the original) columns reflect future ratings assigned 
to the item. In some embodiments, Vouching may also link the newly created column 
to historical ratings. 

When user j Discredits the n ttl item contributed by user i, new columns are 
20 added to the K i and matrices, such that: 

^ +1 = (l-tf) 

where "=* indicates the establishment of a link with column n of the K% and G { 
matrices. The newly created columns reflect the mirror image of future ratings 
assigned to the item. In some embodiments, Discrediting may also link the newly 
25 created column to the mirror image of historical ratings. 

In this embodiment, a user is only permitted to Vouch or Discredit if the user's 
Expertise or Regard (whichever is used in the embodiment), is higher than the 
Expertise or Regard currently associated with the item; initially, this is the Expertise 
or Regard of the user who contributed the item. Each user is permitted to Vouch for or 
30 Discredit a particular item only once. 

The newly created columns would not be factored into the calculation of Quality. 
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13.3 GeHEral Expression 

In some embodiments, a user who Vouches or Discredits might receive only a 
partial link to ratings for the item, expressed generally in the case of Vouching as: 

*? +1 = r(A?) 

and 

9? +l s r(ft») 

And expressed generally in the case of Discrediting as: 

i£' +1 =r(l-tf) 

and 
where 

0 < r < 1 

For example, r = 0.5 would simply divide values in two, so that the user Vouching for 
or Discrediting an item contributed by another user is less affected by ratings. 

15 13.4 Sequential Vouching 

In some embodiments, a user who Vouches or Discredits will be affected by 
ratings on the basis of how much the act raises or lowers the profile of the item, as the 
case may be. 

20 Let U in = {a,b,c,...} b e t h e se t Q f users who have, sequentially beginning with user 
a y Vouched for or Discredited item n contributed by user i, where 

n(hi) < h a , Q(h a ) < h b , Q(h b ) < h c . . . 

where 

if user j Vouches for the item and 

Q(hj) = 1- hj 

25 if user j Discredits the item. 

Let be the m th user in U in . 
In the case of Vouching, let 

Ti ~ h Ux - hi 

and, where m>l 9 

ri = fcif m -n(fc Vm _ l ) 
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In the case of Discrediting, let 

rf = hi - (1 - huj) 

and, where m>\, 

rA = n(/iu m _ 1 )-(l-^ m ) 
5 To include the effect of Vouching in the calculation of Expertise or Regard, let 



and 

To include the effect of Discrediting in the calculation of Expertise or Regard, let 

*£ m+1 = r* (*?) 

10 and 

13.5 Initial Discredit 

In some embodiments, under certain circumstances, users may be permitted or 
15 encouraged to contribute items that are anticipated to yield a negative response. For 
example, an item might be a link to a web site containing objectionable content, 
which the user wishes to bring to the attention of the community of users. In some 
embodiments, a specific forum designed for this purpose may be established, clearly 
separated from other forums. 

20 If such items were to be directly associated with the contributing user, there would 

be substantial disincentives for participation. Therefore, in some embodiments, the 
user contributing the item will have the option to Discredit the item upfront, such that 

Adjusted 

9ii =(1-*S) 

The adjusted rating would be included in calculations of Regard and expertise for the 
25 contributing user. In some embodiments, the adjusted value will also be used in the 
calculation of the Quality of the item. 

13.6 Additional Discussion of Vouching and Discrediting 
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Figs. 20(a^Bid 20(b) are, respectively, a graph ancWrtable to aid in explaining 
the concepts of "Vouching" and "Discrediting" and how these concepts provide a 
solution to the "sparse ratings problem" that arises during the early life of an item. In 
the graph of Fig. 20(a), the vertical axis represents the Regard of an individual posting 
5 an item (e.g., a Regard of 0.400). This is the Regard that is associated with the item, 
for example, for use in the calculation of thread caliber (discussed below in Section 
15). 

In certain embodiments of the present invention, there is a period shortly after 
an article is posted during which any ratings received from users may not be deemed 

10 sufficient to determine a robust measure of item Quality. The term "inertia" in Fig 
20(b) refers to Quality inertia, which may obscure the value of the theretofore 
received ratings, because too few ratings have been received during the period shortly 
following the contribution of the item. During this period, not enough ratings have 
been received to be certain that Basic Quality method or other methods of calculating 

1 5 Quality, or a simple average, or any other measure, is representative of user or expert 
opinion. 

One can, during this period, simply rely on the reputation/Regard of the user 
who contributed the item as a proxy measurement of the value of the item. A 
preferred embodiment, however, permits a user with a higher level of Regard to step 
20 forward and "Vouch" for the item. If, for example, the Regard of the Vouching user 
is 0.600, the Regard associated with the item being Vouched also becomes 0.600. 

When a user Vouches for an item, the user is making the strongest possible 
statement in support of the item in question, backing up that statement with the user's 
reputation. Vouching for an item implies that the user would be willing to take the 
25 ratings offered by other people for the item as if the user had actually authored it. A 
user cannot, for example Vouch a poorly Regarded item half-way up to the user's 
Regard level. Vouching, as used in the described embodiment, is all or nothing, based 
on the Vouching user's Regard value. 

Discrediting is a similarly strong statement. When a user Discredits an item 
30 the user is making the strongest possible statement in opposition to the item in 
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question, aligning HPtull value of the user's reputation agaMf the item. In fact, the 
user would be willing to take the opposite of whatever ratings are given by other 
users. This means that a highly Regarded user with, for example, a Regard of 0.900 
will, by Discrediting an item, reduce the Regard associated with that item to 0.100 
5 (i.e., reduce the associated Regard by 1.000-0.900). In the described embodiment, 
poorly Regarded users are able to Discredit items contributed by highly Regarded 
users. But because items can be Discredited only to the extent of (1 -Regard), a poorly 
Regarded user who Discredits and item will not reduce the Regard associated with the 
item by much. Note that, although Quality values are associated with items and 

10 Regard values are associated with users, the Regard of the author of a particular item 
is often associated with the item. As noted above, for example, in the case of articles 
that have received few ratings, the author's Regard is a helpful benchmark for the 
item's value. The concepts of Vouching and Discrediting allow a "special" Regard to 
be used in calculations involving certain items instead of the Regard of the user who 

15 contributed that item. As shown in the example in the figure, a user having a Regard 
of 0.250 associates this Regard with items by Vouching for the items. Similarly, the 
user gives a Regard of 0.750 to any item which he Discredits. This particular user 
cannot affect the Regard of an item that already has a Regard higher than 0.250 and 
similarly cannot affect the Regard of an item that already has associated with it a 

20 Regard of less than 0.750. 

As items are Vouched for and/or Discredited, the Regard of the contributing 
user, as affected by these procedures, becomes a robust prediction of the Quality that 
will ultimately be calculated for the item. Fig. 20(b) shows calculation of Quality 
values starting at time period 9. (Prior to time period 9, an inertia value preferably is 
25 used for Quality). 

The horizontal axis of Fig. 20(a) shows 15 time periods (1 through 15). 
During each time period, in this example, a user Vouches or Discredits the same item. 
As shown in Figs. 20(a) and 20(b), initially, a user having a Regard of 0.400 posts the 
item. During time periods 2-4, three users Vouch for the item, improving the Regard 
30 associated with the item to 0.850. During time period 5, a user having a Regard of 
0.400 Discredits the item, giving it an associated Regard of 0.600. Users continue to 
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:rHK the item until time penod 15, whenTHsc 



Vouch and DiscflHt the item until time period 15, when^Wser Vouches for the item 
and improves its associated Regard to 0.900. 

In Fig. 20(a), the shaded areas represent the maximum of Regard and Quality 
(i.e., MAX(Regard, Quality)) for the item. As discussed above, a ratings system does 
5 not necessarily have robust information until a certain amount of time has passed 
(e.g., 9 time periods). The results of Vouching and Discrediting, however, provide a 
robust prediction of item Quality much earlier in the process, even when ratings are 
"sparse." 

In addition, sequential Vouching/Discrediting preferably allows the Vouching 
10 or Discrediting user to share in the ratings received for the item, but only to the extent 
that the user moves the current Regard value associate with the item. For example, 
the user Vouching in time period 7 will have the least share in the ratings since the 
user has moved the Regard the least (from 0.800 to 0.875). The user who Discredits 
in time period 12 will have the most significant share (in the opposite of the ratings) 
15 since he moves the Regard the most (from 0.950 to 0.750). 

14. Multi-Item Categories and Attributes 

14.1 Description 

20 In some embodiments, items will fall into separate categories, weighted 

differently in the calculation of Expertise or Regard. Factors considered in 
establishing separate categories will include the difficulty of creating the items, the 
effort required by other users to review the items and the urgency of the items. 

The relative weight attached to an item in calculations of Expertise or Regard will 
25 depend on its category and a number of item attributes that are particular to the 
category. 

14.2 Formulation 

Let b ? 1 designate the category of item n contributed by user i. Let b ? 2 > • • • > b i m 
identify additional attributes of item «, specific to the item's category. 

30 Let the function 

r? = r(6?\...,&r) 
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map the categor^nd attributes of the item to an additionaWveighting factor for use in 
calculations of Regard and expertise. 



14.3 Applied to the Expert Method 

r E Erf*(j ? 5-) 1 



hi = < 



J = ln=l 



£ E r? *g. 



j=ln=l 

otherwise 



14.4 Applied to the High Regard Method 



10 



14.5 



Us 



E Er?Aj*(*3). 

j = ln = l 

E E r?/!?*?. 



j=ln=l 

otherwise 



Example 



Let b i l — 2 , indicating that item 5 contributed by user i is participation in a chat 
session. 





Attributes 


Category 




br 




1 Discussion Forum 


Subject Matter 


Length 


New Thread? 


2 Chat Session 


Participants 


Total Time 


Moderator? 


3 Online Auction 


Bids Submitted 


Image Offered? 


Successfully Sold? 


4 Web Link 


No attributes; all receive the same, fixed 


r" adjustment 


5 Market Forecast 


Equity, Bond, Other? 


How Far Ahead? 


Accuracy 


6 Auction Forecast 


Category 


How Far Ahead? 


Accuracy 



15 



Table 8: Attribute Function 



20 



r i is a function of: 

• the number of participants, b i 2 = 15 

• the total session time in minutes, = 45 and 

• whether or not user i served as a moderator, b i 4 = 0 (indicating not a moderator, 
whereas =1 would indicate the user was a moderator). 



Let 



L52 b 53 

r< = ma *{ i£o> 25 > + max{ Too' * 3} + 6 < 4 * * 2} 
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= 0.15 + 0.3 = 0.45 



15. Caliber Method 



15.1 Description 



Some embodiments will involve the transmission, display or evaluation of 
groupings of items of different Quality, contributed by users with varying Expertise or 
Regard. A good example is a threaded discussion list — two or more discussion 
group postings with a common subject and explicit relationships between messages, 
10 i.e. an indication of which messages respond directly to which other messages. 

One objective of the method is to present groupings containing better items 
first, without breaking apart the thread structure. Multiple threads are sorted among 
themselves, based upon a measurement taking into account characteristics of some or 
all of the contents of each thread. 

1 5 The Caliber method determines C z 9 the "Caliber" of an item grouping. 
15.2 Initial Embodiment 

In the initial embodiment of the method, Caliber is the grouping average of, 
for each item, the higher of Regard (of the user associated with the item) and Quality: 



where z identifies the grouping (e.g. a thread identification number), m the number of 
items contained within, h * the Expertise or Regard of the user who contributed item n 
and <?n the Quality of the item. 



In sorting threads, some embodiments may permit a user to user select ^mm and 9min f 
such that items below a certain level of Quality, or associated with a user of Expertise 
30 or Regard below a certain level, are excluded from the calculation of Caliber: 



1 

C z = — 5^max{J£,ff*} 

771 * ■* 



15.3 



Caliber Threshold 




where 
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15.4 Extended Caliber 

A general formulation for alternative embodiments of the Caliber method is: 

^E$(max{/i^^})l 



5 

based on the same transformation functions discussed above 
Applied to the Caliber threshold method: 



°* W L°«i V ^0 otherwise ' \0 otherwise >) J 



where 



^ [ 1 if /in > h min or q n > h min 
„=i 1 0 otherwise 

10 

16. Auction Method 

Some embodiments will involve the transmission, display and integration of 
Expertise, Regard or Quality into the operation of online auctions. 

The Auction method integrates Expertise or Regard, specifically, into an 
15 auction pricing mechanism. The objective of the method is to increase the integrity, 
fairness and efficiency of online auctions by establishing standards and a procedure 
for a seller to limit the field of users permitted to bid in an auction. 

Let s i be an Expertise or Regard threshold defined by user z, who is putting 
item n up for sale in an online auction. In the initial embodiment, user / selects s ? at 
20 or before the commencement of the auction. If no value is selected, the default is 
zero. 

s ? will establish the minimum Expertise or Regard required for a user to 
submit a binding bid qualified to participate in whatever auction pricing formula 
determines the winner of the auction and the closing price. 
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L^^not meet the s ? standard, in the initi^^^i 



Users who ab not meet the s % standard, in the initiarembodiment, will still be 
permitted to place "record bids" which are bid levels that are recorded and reported, 
either to the seller alone or to the seller and other users, together with data on the 
Expertise or Regard of the user making the record bid, subject to whatever limitations 
5 are in effect on the release of price information pursuant to the auction model. For 
example, in a "second price" style auction, only the second highest bid is disclosed to 
other potential buyers and this standard might be applied to record bids. 

In the initial embodiment, a user placing a record bid will have the option of 
withdrawing the bid at any time before the event described below. 

10 The selling user will have the option at any time until the conclusion of the 

auction, or until a defined period prior to the conclusion of the auction, to reduce the 
value of s ? . Record bids submitted by users who fall within the Expertise or Regard 
threshold after the reduction will immediately be made effective, pari passu with 
already pending bids and the users will no longer be entitled to withdraw. 

1 5 Importantly, although the selling user may reduce s ? , the selling user is not permitted 
to increase its value at any point in the course of the auction. Reductions can be made 
one time, or multiple times during the pendency of an auction. s ? can even be reduced 
to zero. 

In other words, if a seller who begins an auction with strict standards decides 
20 to open up the bidding to users with lesser Expertise or Regard, as the case may be, 
the door is opened to bids from any users that meet the lower standard. The seller will 
not be permitted to discriminate among users according to any standard other than the 
buyer's Expertise or Regard, applied consistently. In alternative embodiments, the 
user's Expertise or Regard may be monitored during the pendency of the auction and, 
25 if the value falls below the current level of s i , the selling user given the option (not 
an obligation) of releasing the user's binding bid. 

17. Data structures 

Fig. 10 shows an example of a small data set in accordance with an 
30 embodiment of the Basic High Regard method. This data set is intentionally small for 
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the sake of exampre^It will be understood that actual data^fcs usually used with the 
methods and systems shown here can be very large. In the example, there are five 
users 1002, each having a user id, such as an e-mail address. Various ones of the 
various users have rated items having item ids 1-13. In the example every user has 
5 authored items. Ratings received from the users for various items vary between a 
lowest rating of 0.03 (items item 7 rated by user 1 and item 4 rated by user 5) and a 
highest rating of 0.95 (items 2 and 3 rated by user 1). Applying the Basic High 
Regard method discussed above yields the Regard (HR) values shown in table 1006 
for each of users 1-5. It will be understood that the Regard values shown in this 
10 example are for purposes of example only and are not to be taken in a limiting sense. 

Fig. 1 1 shows an example of a data structure used to store and retrieve data 
required to perform calculations of Regard and Quality in a preferred embodiment of 
the present invention. This implementation uses circular linked lists to represent the 
sparse matrixes used to store the ratings contributed by the users. The Figure shows a 
15 Users linked list 1 100 and an Items linked list 1 140. Users list 1 100 contains an entry 
for p+1 users. The users have user IDs (uids) 0 through p. Items list 1 140 contains an 
entry for n+1 items that were contributed by the users. The items have item IDs (iids) 
0 through n. 

Each element in Users list 1100 has a corresponding Authors linked list 1120. 

20 In the Figure, only one Author list is shown to improve the clarity of the Figure. The 
entry in Users list 1 100 for user 2 has an associated Author listl 1 120 containing three 
entries. These entries in the Author list 1120 shown in the Figure represent the 
authors whose items have been rated by user 2. Thus, the first entry 1 122 represents 
user 1, whose item(s) were rated by user 2; the second entry 1124 represents user 4, 

25 whose item(s) were rated by user 2; and the third entry 1 126 represents user 7, whose 
item(s) were rated by user 2. 

Each element in an Authors list 1120 has a corresponding Ratings linked list 
1130. In the Figure, only two Ratings list are shown for the sake of clarity. For 
example, the entry 1122 in Authors list 1120 for Author 1 has an associated Ratings 

30 list 1130 containing three entries. These entries in the Ratings lists 1130 represent 
items contributed by User 1 (and rated by user 2). Thus, the first entry 1132 
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represents item 3^wnich was contributed by user 1 and nred by user 2; the second 
entry 1 134 represents item 8, which was contributed by user 1 and rated by user 2; the 
third entry 1 136 represents item 17, which was contributed by user 1 and rated by user 
2. 



author (aid = author's uid), a "k" value and an "m" value. Each "g" value in a Ratings 
list 1 130 represents a rating for the item. The "r" value in each entry of Authors list 
1 120 is the sum of the ratings g from its corresponding Ratings list 1 130. Thus, for 
example, the sum of the ratings g in entries 1132, 1134, 1136 is stored as value r in 
10 entry 1122. 

Each entry in a Ratings list 1130 has a "k" value. In the example, k=l, 
although k can also have other values. The "t" value in each entry of Authors list 
1 120 is the sum of the "k" values from its corresponding ratings list 1 130. Thus, for 
example, the sum of the "k" values in entries 1132, 1134 and 1136 is stored as value 
15 "fin entry 1122. 

In the described embodiment, when author 2 rates an item of author 1, the 
rating is added to the Ratings list 1 130 for Author 1 and the "t" value for Author 1 is 
incremented. 



20 Regard values) and a plurality of circular linked lists 1160 (used to find "Quality"). 
Only one of each of lists 1150 and 1160 is shown for the sake of clarity. Each list 
1 150 is formed by a series of b_next links and represents the "r" and "t" values of a 
single author. Thus, in the Figure, list 1 150 contains entries whenever items authored 
by user 1 were rated by another user. Thus, entry 1122 represents items of user 1 

25 rated by user 2. Entry 1154 represents items of user 1 rated by user 0. In the 
example, entry 1154 points to the entry for user 1 in Users list 1100, which points in 
turn to entry 1 122, forming circular linked list 1 150. To calculate "high Regard," the 
method follows the b_next chain for a list 1150 to pick up the non-zero "r" and "t" 
values for a particular author. 

30 Each list 1160 is formed by a series of s_next links and represents the ratings 

"g" of a single item that has been rated by various users. Thus, in the Figure, list 1 160 
contains entries whenever item 2 of user 4 was rated by another user. Thus, entry 



5 



Each entry in a Ratings list 1130 includes 



a "g" value, an author ID of the 



The Figure includes a plurality of circular linked lists 1 150 (used to find high 
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1 162 represents it 




of user 4 rated by user 2. Entry 1 1 




►resents item 2 of user 



4 rated by user 9. In the example, entry 1 164 points to entry 1 142 in Items list 1 140 
(for item 2). Entry 1142 points to entry 1162 , forming a circular linked list. To 
calculate "Quality" for an item, the method follows the s_next chain for a list 1 160 to 
5 pick up the non-zero ratings "g" values. 

As shown, each user in User list 1100 has a corresponding "Regard" in high 
Regard list 170. Each entry in the lists is preferably time-stamped (to aid, for 
example, in the decay method discussed above). 



10 discussion group postings, participation in an online chat session, the products, 
services and assets one offers to sell in online auctions and web link 
recommendations, etc. can be considered items of different types. The concept of 
"multi-type High Regard" is created so that an item of one type may have more (or 
less) influence on the High Regard values than an item of a different type. 



18. Example Architectures 

It will be understood that the example architectures discussed herein are 
provided by way of example and are not to be taken in a limiting sense. There are a 
large number of possible architectures and network structures that benefit from having 

20 a server such as rating server 140 of Fig. 1 therein (or a similar rating server returning 
Expertise and Quality). In the figures that follow, the function of receiving ratings 
and user information and returning Quality and/or Regard (Expertise) information is 
preferably performed by rating server 1202. It will also be understood that rating 
server 1202 can be part of a larger data processing system performing one or more of 

25 the functions discussed herein. Rating server 1202 is shown separately, but is not 
required to be separate from the entities to which it provides information. In addition, 
the functions of rating server 1202 and the other entities discussed below can be 
distributed over more than one data processing system or network without departing 
from the spirit and scope of the present invention. 

30 Fig. 12(a) is a block diagram of a first example forum server application 

including a rating server 1202. A forum server 1204 provides, for example, the 



There may be situations where different types of items co-exist. For example, 



15 
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content of a forunMBch as that shown in Fig. 2(a). FonJHPserver 1204 accesses a 
database 1206 storing the items displayed by the forum and preferably caching the 
Quality and Regard values returned from rating server 1202, although not all servers 
1204 cache. Expertise and/or Caliber values might also be returned. Rating server 
5 1202 accesses its own database 1208 containing information about the ratings, Quality 
and the Regard of the various users (including associated Regards caused by 
Vouching and Discrediting, if these features are part of the system). For example, the 
data structure of Fig. 11 is preferably stored in database 1208. It will be understood 
that the databases 1206, 1208 shown can be located at other appropriate places in the 

10 system without departing from the spirit and scope of the invention. In Fig. 12(a), a 
user requests forums, threads and articles within the threads and posts his own items 
through interaction with forum server 1204. Forum server 1204 interacts with rating 
server 1202 to obtain the Regard of the contributing users and the item Quality 
requested. Forum server 1204 also sends ratings contributed by users to rating server 

15 1 202 as they are received. 

Thus, forum server 1204 identifies new items contributed by the users to rating 
server 1202. The items them selves are not necessarily sent to server 1202, but server 
1202 need to know that new items have been contributed. An item ID can be 
determined either by forum server 1204 (in which case, the item ID is preferably 
20 stored in conjunction with a forum ID to uniquely identify the item) or by rating 
server 1202. Forum server 1202 also identifies the existence of new authors to rating 
server 1202. Again, an author id can be established either by forum server 1204 or by 
rating server 1202. 

Forum server 1204 also identifies the existence of new users to rating server 
25 1202. A user id can be established either by forum server 1204 or by rating server 
1202. 

Forum server 1204 also identifies the existence of new ratings to rating server 
1204. These ratings are received by forum server 1204 from users via an interface 
such as that shown in Fig. 2. In alternative embodiments, ratings are received directly 
30 from the users without having to go through forum server 1204. Communication 
between servers 1202 and 1204 and between the servers and user 1201 can be 
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accomplished usm^any appropriate protocol and mess^^ format. For example, 
rating and forum items can be sent to a user's browser using the well-known http Web 
protocol. Similarly, information can be exchanged among the elements of Fig. 12(a) 
using any appropriate non-web protocol. 

5 Fig. 12(b) shows another embodiment of servers 1202 and 1204 in which 

server 1202 communicates directly with the user's browser instead of with forum 
server 1204. In this embodiment, server 1204 still sends information about new users, 
new items, new ratings, etc to rating server 1202, but requests for Quality, Expertise, 
Caliber and/or Regard values are sent by a browser of user 1201 and returned by 
10 rating server 1202 directly to the user's browser. The browser may cache these values 
in certain embodiments. 

Fig. 12(c) shows an example of a web page where the html of the page cause a 
web page including items (or descriptions of items) to be fetched from forum server 
1204 and Regard/Expertise/Caliber and/or Quality values to be fetched separately 
15 from server 1202. Alternately (not shown) a web page could be fetched from a third 
server. This third party web page might include links to both forum server 1204 and 
1202. When the browser encounters these links on the web page, it requests 
information from the specified server and incorporates it into the displayed web page. 

Fig. 13 is a block diagram of another example forum server application 
20 communicating with a separate integrated content server 1302 and a rating server 
1202. Integrated content server 1302 communicates with forum server 1204 to obtain 
items (e.g., threads of forum messages) and communicates with forum server 1204 
(and indirectly with rating server 1202) to obtain Regard/Expertise for users and/or 
Quality for items. Server 1302 preferably also communicates with a global network. 
25 For example, integrated content server 1302 might obtain content from an outside 
source (for example, an online newspaper) and add ratings information to the content 
so obtained. 

Fig. 14 is a block diagram of another example forum server application 1204 
communicating with a separate integrated content server 1302 and a rating server 
30 1202. An ad server 1402 can be, for example, a known commercial ad server, such as 
the Doubleclick, 24/7 or Adforce ad servers. Ad server 1402 communicates with a 
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o^HTample, to deliver ad content. The us^W4 



user's browser, fd^STample, to deliver ad content. The us^W401 views the contents 
of a web page including ads from ad server 1404 and integrated content from server 
1302. The integrated content includes data from forum server 1204 and Regard 
and/or Quality values from rating server 1202. The ads can include clickthrough 
5 banners from ad server 1402 and/or advertiser's web site 1404. Servers 1402/1404 
access a commercial data server 1406. Commercial data server 1406 can track the 
user's identity through information provided from servers 1402 and 1404 and obtains 
the Regard and/or item Quality from rating server 1202. Commercial data server 
1406 then provides the obtained ratings information to the requesting advertiser web 
10 site 1404, ad server 1402 and forum server 1204 to help them target their advertising. 
Thus, rating server 1202 provides data both to the forum server 1204 and to 
commercial data server 1406, which forwards the data to its own requesting clients. 

Alternatively, rating server 1202 can provide ratings directly to the browser of 
user 1401 in a manner similar to that shown in Fig. 12(b). 

15 Fig. 15(a) is a flow diagram showing communication between elements of Fig. 

14 during forum/thread index intercommunication. This flow is similar to that shown 
in Fig. 13, except that Fig. 13 does not include an ad server that provides the user with 
ads on the web pages the user is viewing. Details of this process are shown in Figs. 
16(a)- 16(c). 

20 Fig. 15(b) is a flow diagram showing communication between elements of Fig. 

14 during article view intercommunication. Here, the user requests a particular 
article/item in a current thread. The forum server requests ratings for the article from 
rating server 1202 and provides the items and ratings to the user. When the user rates 
an item, the rating is sent to rating server 1202, which calculates a new Quality for the 

25 item and sends the new Quality to the forum server and web server with instructions 
that the new Quality invalidates any previously cached Quality for the item. Details 
of this process are shown in Figs. 16(d)- 16(f). 

Fig. 15(c) is a flow diagram showing communication between elements of Fig. 
14 during post/reply intercommunication. When the user posts a new article in a 
30 thread, the article/item is sent to rating server 1202. Rating server 1202 sends a 
message to web server 1302 to invalidate any cached Caliber values for the thread (as 
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i^Khe thread). The forum server may hf^^e 



opposed to itemsTMie thread). The forum server may ha^He the caliber calculation. 
Alternatively, caliber may be calculated by a Java applet on the user's own machine. 
Details of this process are shown in Figs. 16(g)- 16(h). 

Fig. 17 is a block diagram of an example integrated content server 1302, 
5 where the user's browser also communicates with an e-commerce web site 1702 and 
an e-commerce sourcing server 1704. Rating server 1202 provides Quality and/or 
Regard values to forum server 1204 and to commercial data server 1406. The user 
visits the e-commerce web site, which communicates the user's identity to 
commercial data server 1406. As in Fig. 14, server 1406 communicates with rating 
10 server 1202 to obtain information about the user and passes it to its own requesting 
clients, such as site 1702, server 1704, and server 1204. 

Fig. 18 is a block diagram of an example auction server 1802, where the user's 
browser also communicates with an e-commerce web site 1702 and an e-commerce 
sourcing server 1704. Rating server 1202 provides Quality and/or Regard values to 
15 forum server 1204 and to commercial data server 1406. The user visits the e- 
commerce web site, which communicates the user's identity to commercial data 
server 1406. As in Fig. 14, server 1406 communicates with rating server 1202 to 
obtain information about the user and passes it to its own requesting clients, such as 
site 1702, server 1704, and server 1204. 

20 Alternatively, the functionality of rating server 1202 is integrated into auction 

web server 1802, so that auction server 1202 calculates Regard and Quality and 
receives ratings. Thus, auction web server 1802 can provide Quality and Regard 
information about items on the auction site and about auctions transactions as 
discussed above. Alternatively, rating server 1202 communicates Regard and/or 

25 Quality information to respective browsers of users 1810 

Fig. 19 is a block diagram of an example rating server 1202 for an individual 
and commercial rating service. Here, rating server 1202 communicates with browser 
or communication software of user 1902 to provide that user with ratings (Quality 
and/or Regard) of other users and their items. Rating server 1202 also communicates 
30 with commercial data server 1402 as described above in connection with Fig. 14. The 
commercial data server passes information concerning ratings obtained from rating 
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server 1202 to bS^-commerce web sites and advertiser^?eb sites and to still other 
individual users. Thus, rating server 1202 provides data both to individuals and to a 
commercial service in this example. 

Other embodiments will be apparent to those skilled in the art from consideration 
of the specification and practice of the invention disclosed herein. It is intended that 
the specification and examples be considered as exemplary only, with a true scope of 
the invention being indicated by the following claims and equivalents. 
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